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DOD MAJOR AUTOMATED INFORMATION 
SYSTEMS 

Improvements Can Be Made in Applying Leading 
Practices for Managing Risk and Testing 


What GAO Found 

Most of the 18 selected Department of Defense (DOD) major automated 
information system (MAIS) programs that GAO reviewed had experienced 
changes in their planned cost and schedule estimates and half of the programs 
had met their technical performance targets. Specifically, 16 programs 
experienced changes in their cost estimates ranging from a 39 percent decrease 
($1.47 billion) to a 469 percent increase ($1.63 billion). The average cost 
increase was $457.2 million among the 11 programs reporting an increase. 
Fourteen programs experienced schedule delays, which ranged from 2 months 
to over 13 years. Finally, half of the MAIS programs fully met all of their technical 
performance targets. Of the remaining nine programs, 4 four had partially met 
their target because each was still conducting tests. The other five programs 
were in the early stages of system development and had not begun testing. 

In addition, for the five MAIS programs GAO selected for in-depth review, all had 
either fully or partially applied leading practices for managing requirements, risks, 
and for conducting systems testing and integration. 

• Managing requirements. Three of the five programs had fully implemented 
the practices for managing requirements, while the other two had partially 
implemented some practices. Leading practices in this area include 
establishing requirements and ensuring traceability between requirements 
and work products. 

• Managing risks. Three of the five programs had fully implemented the risk 
management practices, while two had partially implemented some practices. 
An effective risk management process identifies potential problems before 
they occur. For example, one Army program did not have standard operating 
procedures for defining thresholds or bounds to manage risk. Unless such 
procedures are defined, the program will not have the tools needed to define 
risk management activities, including whether and how certain risks are 
prioritized. Further, programs should include practices to identify potential 
problems so that risk-handling activities may be planned and invoked across 
the project to mitigate the potential for adverse impacts. However, one Air 
Force program did not develop an overall risk mitigation plan to guide the 
implementation of individual risk mitigation activities. Without an overall risk 
plan to guide individual development efforts, those efforts cannot be 
managed cohesively. 

• Testing and integration. Four of the five programs had fully implemented 
practices for systems testing and integration. Programs should, among other 
activities, establish roles and responsibilities to manage testing and 
integration activities, including a chief developmental tester to oversee 
testing activities. However, one Air Force program reported difficulty in hiring 
a qualified individual to perform these duties. Until this position is filled, the 
program may not effectively manage risks and verify compliance with system 
acquisition and operational requirements. 
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441 G St. N.W. 
Washington, DC 20548 


U.S. GOVERNMENT ACCOUNTABILITY OFFICE 


March 30, 2017 
Congressional Committees 

The Department of Defense (DOD) is one of the largest and most 
complex organizations in the world. To meet its mission to protect the 
security of our nation and to deter war, it relies heavily on the use of 
information technology (IT) to support our warfighters. In this regard, 
according to DOD’s IT investment portfolio for fiscal year 2016, the 
department spent approximately $31 billion for such investments. 1 Of this 
amount, approximately $2.5 billion was spent on major automated 
information system (MAIS) programs, which include systems that help the 
department sustain its key operations in communications, business, and 
command and control, and provide the department with access to 
information to organize, plan, direct, and monitor mission operations. 

A DOD IT acquisition investment that exceeds one of the following cost 
thresholds is designated as a MAIS program: (1) program costs in any 
single year exceed $40 million, (2) total program acquisition costs exceed 
$165 million, or (3) total life-cycle costs exceed $520 million in fiscal year 
2014 constant dollars. The Secretary of Defense can also use discretion 
to designate a program as a MAIS if it does not meet these cost 
thresholds. 

The National Defense Authorization Act for Fiscal Year 2012 includes a 
provision that we select, assess, and report on DOD MAIS programs 
annually through March 2018. 2 This report is the fifth in a series of annual 
assessments. 3 Our objectives for this review were to (1) describe the 


1 DOD’s IT investment portfolio identifies all of its IT investments and associated costs 
within the department and its components. 

Effective September 30, 2017, the statutory major automated information system 
provisions are repealed. National Defense Authorization Act for Fiscal Year 2017, Pub. L. 
No. 114-328, § 846 (Dec. 23, 2016). 

3 GAO, DOD Major Automated Information Systems: Improvements Can Be Made in 
Reporting Critical Changes and Clarifying Leadership Responsibility, GAO-16-336 
(Washington, D.C.: March 30, 2016); Defense Major Automated Information Systems: 
Cost and Schedule Commitments Need to Be Established Earlier, GAO-15-282 
(Washington, D.C.: February 26, 2015); Major Automated Information Systems: Selected 
Defense Programs Need to Implement Key Acquisition Practices, GAO-14-309 
(Washington, D.C.: March 27, 2014); Major Automated Information Systems: Selected 
Defense Programs Need to Implement Key Acquisition Practices, GAO-13-311 
(Washington, D.C.: March 28, 2013). 
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extent to which selected MAIS programs had changed their planned cost 
and schedule estimates, and met technical performance targets and (2) 
assess the extent to which selected MAIS programs have used leading IT 
acquisition practices, including requirements and risk management, and 
systems testing and integration. 

To address the first objective, we used DOD’s official list of 35 MAIS 
programs, as of May 1,2016, to establish a basis for selecting programs. 
We selected 18 of 35 MAIS programs based on our criteria that programs 
must be unclassified and have a first acquisition program baseline that 
could be used as a reference point for evaluating cost, schedule, and 
technical performance characteristics. 

We then compared each program’s cost (in then-year dollars) and 
schedule estimates established in the first acquisition baseline to the 
latest total life-cycle cost and schedule estimates. 4 In addition, to 
determine whether technical performance targets 5 were met, we reviewed 
each program’s initial and the most recent baseline targets. We 
summarized the results of our analyses for each individual MAIS 
program. 

To address the second objective, we selected 5 programs for a more in- 
depth review from among those used by at least two military services and 
that had not been included in a GAO assessment in the past year to focus 
on programs that have not been recently evaluated. Using these criteria, 
we selected the 5 programs listed in table 1. 


4 Then-year dollars are those dollars that include the effects of inflation or escalation 
and/or reflect the price levels expected to prevail during the year at issue. The first 
acquisition program baseline is established after the program has assessed the viability of 
various technologies and refined user requirements to identify the most appropriate 
technology solution that demonstrates that it can meet users’ needs. The Defense 
Acquisition Guidebook (which complements and further explains DOD’s acquisition 
policies and process) refers to a program’s best cost and schedule estimates as objective 
estimates. 

5 MAIS annual reports include information on the status of each technical performance 
target, among other information. Information about each target includes a description of 
the performance characteristics, the objective and threshold value for each target and, 
importantly, whether the target has demonstrated the expected performance. 
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Table 1: MAIS Programs Assessed Against Leading Practices 

Program name 
Army 

Global Combat Supply System Increment 1 (GCSS-A Inc 1) 

Integrated Personnel and Pay System Increment 2 (IPPS-A Inc 2) 

Air Force 

Air and Space Operations Center-Weapon System Increment 10.2 (AOC-WS Inc 10.2) 
Base Information Transport Infrastructure Wired (BITI Wired) 

Joint Space Operations Center, Mission System Increment 2 (JMS Inc 2) 

Source: DOD data | GAO-17-322 


To evaluate DOD’s approach in managing the acquisition of MAIS 
systems, we identified risk management and requirements management 
practices from the Software Engineering Institute’s Capability Maturity 
Model® Integration for Acquisition (CMMI-ACQ), and assessed each of 
the programs against these leading practices. 6 With regard to risk 
management, we analyzed each program’s key documents, such as risk 
management plans, risk charts, and other artifacts, and compared them 
to the leading practices. For requirements management, we compared 
requirements documents, such as the requirements management plan, 
traceability matrix, and procedural tools to the leading practices. 7 

We also identified key systems testing and integration practices from the 
Institute of Electrical and Electronics Engineers, Incorporated (IEEE), 8 


6 Software Engineering Institute, Capability Maturity Model® Integration for Acquisition 
(CMMI-ACQ), Version 1.3 (Pittsburgh, Pa.: November 2010). 

7 CMMI-ACQ; and GAO, GAO Cost Estimating and Assessment Guide: Best Practices for 
Developing and Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: 

March 2009); Information Technology: DHS Needs to Improve Its Independent Acquisition 
Reviews, GAO-11-581 (Washington, D.C.: July 28, 2011); Information Technology: Critical 
Factors Underlying Successful Major Acquisitions, GAO-12-7 (Washington, D.C.: Oct. 21, 
2011); and Schedule Assessment Guide: Best Practices for Project Schedules, 

GAO-16-89G (Washington, D.C.: December 2015). 

8 The Institute of Electrical and Electronics Engineers, IEEE International Standard for 
Software and Systems Engineering, IEEE Standard 29119-1-2013™ (New York, NY: 

Sept. 1,2013). All rights reserved. 
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and DOD’s policy and guidance 9 (DODI 5000.02, Defense Acquisition 
Guidebook, and Test & Evaluation Management Guide) and assessed 
each of the five programs against these practices. For example, we 
evaluated the test manager roles and responsibilities, test plans, and test 
reports to determine the extent to which these practices were applied. We 
interviewed program officials to obtain additional information on each 
program’s risk and requirements management processes, and systems 
testing and integration practices. 

We conducted this performance audit from April 2016 to March 2017 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. See appendix I for a more 
detailed discussion of our objectives, scope, and methodology. 


Background 


DOD’s organizational structure includes the Office of the Secretary of 
Defense, the Joint Chiefs of Staff, the military services, numerous 
defense agencies and field activities, and various unified combatant 
commands that contribute to the oversight of DOD’s MAIS programs. 
Figure 1 provides a simplified depiction of DOD’s organizational structure. 


department of Defense, Operation of the Defense Acquisition System, DODI 5000.02 
(January 7, 2015); Department of Defense, Defense Acquisition Guidebook ; Department 
of Defense, Test and Evaluation Management Guide 6th Edition (Washington, D.C.: 
December 2012). 
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Figure 1: Simplified DOD Organizational Structure 



Source: GAO analysis based on Department of Defense data. | GAO-17-322 


The Under Secretary of Defense for Acquisition, Technology, and 
Logistics (USD (AT&L)) serves as the Defense Acquisition Executive and 
is the official responsible for supervising the acquisition and oversight of 
MAIS programs. 10 This official is the principal acquisition official of the 
department and is the acquisition advisor to the Secretary of Defense. 
The USD (AT&L) has policy and procedural authority for the defense 
acquisition system, which establishes the steps that DOD programs 
generally take as the department plans, designs, acquires, deploys, 
operates, and maintains its IT systems (discussed in more detail following 
this section). The USD (AT&L)’s authority also includes directing the 
military services and defense agencies on acquisition matters and— 
unless the authority is delegated to the service or agency—making 
milestone decisions for MAIS programs. 


10 Effective February 1,2018, the position of Under Secretary of Defense for Acquisition, 
Technology and Logistics will be eliminated and the newly created positions of Under 
Secretary of Defense for Research and Engineering and Under Secretary of Defense for 
Acquisition and Sustainment will incorporate duties from the eliminated position. This 
change is per the National Defense Authorization Act for Fiscal Year 2017, Pub. L. No. 
114-328, § 901 (Dec. 23, 2016). 
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DOD’s Acquisition 
Guidance and Framework 
for Managing MAIS 
Acquisitions 


The Department of Defense Instruction 5000.02 11 is the policy that 
establishes guidelines for the management of all acquisition programs. 
The department’s policy guidance includes six defense acquisition 
models based on the type of product being acquired (e.g., software¬ 
intensive programs and hardware-intensive programs). A generic 
acquisition model that shows all of the program life-cycle phases and key 
decision points is shown in figure 2 and described below. 


Figure 2: Generic Model from the Defense Acquisition System Framework 
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Source: GAO analysis based on Department of Defense data. | GAO-17-322 


Materiel solution analysis: Refine the initial system solution (concept) 
and create a strategy for acquiring the solution. A decision—referred to as 
Milestone A, the risk reduction decision point—is made at the end of this 
phase to authorize entry into the technology maturation and risk reduction 
phase. 


Technology maturation and risk reduction: Determine the preferred 
technology solution and validate that it is affordable, satisfies program 
requirements, and has acceptable technical risk. A decision—referred to 
as Milestone B, the program initiation decision point—is made at the end 
of this phase to authorize entry of the program into the engineering and 
manufacturing development phase and award development contracts. An 


^DOD Instruction 5000.02, Operation of the Defense Acquisition System, (Washington, 
D.C.: Jan. 7, 2015). 
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acquisition program baseline 12 is first established at the Milestone B 
decision point. A program’s first acquisition program baseline contains the 
original life-cycle cost estimate (acquisition and operations and 
maintenance costs), the schedule estimate (major milestones and 
decision points), and performance parameters that were approved for that 
program by the Milestone Decision Authority. 13 The first acquisition 
program baseline is established after the program has refined user 
requirements and identified the most appropriate technology solution that 
demonstrates that it can meet users’ needs. 

Engineering and manufacturing development: Develop a system and 
demonstrate through testing that the system meets all program 
requirements. A decision—referred to as Milestone C, the deployment 
decision point—is made during this phase to authorize entry of the 
system into the production and deployment phase or into limited 
deployment in support of operational testing. 

Production and deployment: Achieve an operational capability that 
meets program requirements, as verified through independent operational 
tests and evaluation, and implement the system at all applicable 
locations. 

Operations and support: Operationally sustain the system in the most 
cost-effective manner over its life cycle. 


12 The acquisition program baseline is developed by the program manager and approved 
by the Milestone Decision Authority (the official responsible for deciding whether a 
program meets the milestone criteria and proceeds to the next phase of the acquisition 
process) before the formal initiation of all acquisition programs and describes the 
parameters against which program performance will be measured and tracked. The 
acquisition program baseline states the threshold and objective values for the cost, 
schedule, and performance requirements fora program. An acquisition program baseline 
is required for each increment and block for a Major Defense Acquisition Program. Any 
deviations from an approved acquisition program baseline must be documented. 

13 Depending on the program, the Milestone Decision Authority can be the Under 
Secretary of Defense (Acquisition, Technology, & Logistics), the head of the relevant 
Department of Defense component, or the component acquisition executive. 
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Leading Practices for 
Managing an IT 
Acquisition Program 


Entities such as the Project Management Institute, the Software 
Engineering Institute at Carnegie Mellon University, IEEE, and GAO have 
developed and identified leading practices to help guide organizations to 
effectively plan and manage the acquisition of major IT systems, such as 
the MAIS programs. Our work has shown that proper implementation of 
such practices can significantly increase the likelihood of delivering 
promised system capabilities on time and within budget. These practices 
include, but are not limited to, 

• Risk management: A process for anticipating problems and taking 
appropriate steps to mitigate risks and minimize their impact on 
program commitments. It involves identifying and documenting risks, 
categorizing them based on their estimated impact, prioritizing them, 
developing risk mitigation strategies, and tracking progress in 
executing the strategies. 

• Requirements management: Requirements establish what the 
system is to do, how well it is to do it, and how it is to interact with 
other systems. Appropriate requirements development involves 
eliciting and developing customer and stakeholder requirements, and 
analyzing them to ensure that they will meet users’ needs and 
expectations. It also consists of validating requirements as the system 
is being developed to ensure that the final systems to be deployed will 
perform as intended in an operational environment. 

• Systems testing and integration management: A rigorous and 
efficient test and evaluation program provides early knowledge of 
developmental and operational issues. Correcting these issues early 
can mitigate risks of cost overruns and schedule slippages, and can 
ultimately contribute to delivery of effective and suitable systems in a 
timely manner. System integration ensures that all the “pieces” of the 
system can work together to realize the program’s goals. It involves 
the collaborative planning and execution of test phases and events to 
provide shared data in support of independent analysis, evaluation, 
and reporting by all stakeholders. 
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Most Selected MAIS 
Programs Had 
Changes in Their 
Planned Cost or 
Schedule Estimates, 
and Half Met Their 
Technical 

Performance Targets 


The majority of the 18 selected MAIS programs that we reviewed had 
experienced changes in their planned cost estimates and in their planned 
schedule estimates when comparing the first acquisition program 
baseline to the most recent acquisition program baseline estimate. 14 
These cost estimate changes ranged from a decrease of $1,466.2 million 
(-39 percent) to an increase of $1,625.7 million (469 percent), and 
schedule estimate changes ranged from a slippage of 13 years and 9 
months to no change. Further, nine of the programs had met all of their 
technical performance targets, while four of the programs had partially 
met the performance targets. The remaining five programs had not yet 
conducted testing activities. As a result, it is too early to report on the 
status of technical performance targets. Table 2 shows the extent of 
changes in planned cost and schedule estimates for each of the selected 
MAIS programs we reviewed since the first program baseline estimate 
and the number of technical performance targets met. See appendix II for 
the detailed profiles of each program. 


Table 2: Summary of Changes in Cost and Schedule Estimates and the Status of Technical Performance Targets for the 18 
Selected Major Automated Information System Programs (from the first acquisition baseline estimate) 

Program name 

Change in 
cost estimate 
(dollars in 
millions) 

Change in 
schedule estimate 
(delay) 

Number of 
performance 
targets met 

Army 

• Tactical Mission Command (TMC) 

$1,421.7 (159%) 

3 months 

3 of 3 

• Integrated Personnel and Pay System-Army Increment 1 
(IPPS-AIncI) 

$10.5 (3%) 

6 months 

3 of 3 

• Integrated Personnel and Pay System-Army Increment 2 
(IPPS-A Inc 2) 

$330.9 (19%) 

No change 

Tests not yet 
conducted 

• Logistics Modernization Program Increment 2 (LMP Inc 2) 

$6.9 (1 percent) 

5 months 

7 of 7 

• Global Combat Support System-Army (GCSS-A Inc 1) 

-$58.5 (-1.5%) 

10 months 

2 of 2 

Air Force 

• Defense Enterprise Accounting and Management System 
Increment 1 (DEAMS Inc 1) 

$124.9 (9%) 

1 year 

6 of 9 


14 

The acquisition program baseline is developed by the program manager before the 
initiation of a program for all acquisition programs and depicts the current condition of a 
program. The plan (which must be approved by the Milestone Decision Authority) states 
the threshold and objective values for the cost, schedule, and performance requirements 
fora program. An acquisition program baseline is required for each increment and block 
for a MAIS or Major Defense Acquisition Program. Any deviations from an approved 
acquisition program baseline must be documented. 


Page 9 


GAO-17-322 DOD Major Automated Information Systems 




Program name 

Change in 
cost estimate 
(dollars in 
millions) 

Change in 
schedule estimate 
(delay) 

Number of 
performance 
targets met 

• Integrated Strategic Planning and Analysis Network 

Increment 4 (ISPAN Inc 4) 

No change 

No change 

Tests not yet 
conducted 

• Air and Space Operations Center-Weapon System Inc 10.2 
(AOC-WS Inc 10.2) 

$56.8 (1%) 

3 years 

Tests not yet 
conducted 

• Joint Space Operations Center Mission System Increment 2 
(JMS Inc 2) 

-$134.6 (-12%) 

2 years 10 months 

Tests not yet 
conducted 

• Base Information Transport Infrastructure Wired (BITI Wired) 

-$1,466.2 (-39%) 

2 months 

Oof 4 a 

Navy 

• Common Aviation Command and Control System Increment 

1 (CAC2S Inc 1) 

$1,625.7 (469%) 

13 years 9 months 

2 of 2 

• Global Combat Support System-Marine Corps Logistics 

Chain Management Increment 1 (GCSS-MC LCM Inc 1) 

$1,360.3 (295%) 

5 years 7 months 

2 of 2 

• Consolidated Afloat Network Enterprise Services (CANES) 

-$830.3 (-7%) 

2 years 

8 of 9 

Defense Information Systems Agency 

• Teleport Generation 3 (Teleport Gen 3) 

$11.6(2%) 

3 years 2 months 

1 of 3 

Defense Health Agency 

• Department of Defense Healthcare Management System 
(DHMSM) 

No change 

No change 

Tests not yet 
conducted 

• integrated Electronic Health Record Increment 1 
(iEHR Inc 1) 

-$718.3 (-70%) 

11 months 

3 of 3 

Defense Logistics Agency 

• Defense Agencies Initiative Increment 2 (DAI Inc 2) 

$2.6(1%) 

No change 

5 of 5 

Department of Defense 

• Theater Medical Information Program-Joint Increment 2 
(TMIP-J Inc 2) 

$374.9 (161%) 

5 years 6 months 

10 of 10 

Source: GAO analysis of data provided by DOD officials | GAO-17-322 




a We categorized BITI Wired as partial because it has met 4 of 4 targets at 167 of 178 total sites. 


Of the 18 MAIS programs we reviewed, 16 had experienced changes in 
their planned cost estimates. Specifically, five of the 16 programs had 
cost decreases that ranged from $58.5 million (-1.5 percent) for the 
Army’s Global Combat Support System, Increment 1 (GCSS-A Inc 1) 
program to $1,466.2 million (-39 percent) for the Air Force’s Base 
Information Transport Infrastructure Wired (BITI Wired) program. 

Officials reported various reasons for the decreases in planned cost 
estimates. For example, the Defense Health Agency’s integrated 
Electronic Health Record Increment 1 (iEHR Inc 1) program reported that 
the 70 percent cost decrease was due to a reduction in the program’s 


Sixteen of Eighteen 
Selected MAIS Programs 
Had Changes in Their 
Planned Cost Estimates 
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scope. Officials for the Air Force’s BITI Wired program reported that the 
39 percent cost decrease was due to early assumptions about risks not 
materializing. For example, when the program had solicited bids for the 
contract, they included risk factors that never materialized. 

In addition, 11 of the 16 programs had experienced cost increases. These 
increases ranged from $2.6 million (1 percent) for the Defense Logistics 
Agency’s Defense Agencies Initiative Increment 2 (DAI Inc 2) program to 
$1,625.7 million (469 percent) for the Navy’s Common Aviation Command 
and Control System Increment 1 (CAC2S Inc 1) program. Among the 11 
programs that reported an increase, the average increase was $457.2 
million. 

Program officials reported a variety of reasons for the increases in 
planned cost estimates. For example, officials for the Army’s Tactical 
Mission Command (TMC) program attributed its 159 percent increase to, 
in part, higher than expected costs for research and development testing. 
In addition, officials for the Air Force’s Defense Enterprise Accounting and 
Management System Increment 1 (DEAMS Inc 1) program attributed a 9 
percent cost increase to the program’s growth in scope and the addition 
of software upgrade enhancements. Further, officials for the Navy’s 
CAC2S system attributed cost increases of 469 percent to the program’s 
early developmental challenges, which included program scope growth 
and restructuring. 

Two programs had not experienced any changes in their cost estimates: 
These include the Defense Health Agency’s Department of Defense 
Healthcare Management System (DHMSM) program and the Air Force’s 
Integrated Strategic Planning and Analysis Network Increment 4 (ISPAN 
Inc 4) program. 


Fourteen of Eighteen 
Selected MAIS Programs 
Had Changes in Their 
Planned Schedule 
Estimate 


Of the 18 selected MAIS programs we reviewed, 14 programs had 
experienced changes in their planned schedule estimates. These 
changes consisted of schedule slippages that ranged from 2 months for 
the Air Force’s BITI Wired program to 13 years and 9 months for the 
Navy’s CAC2S program. 

Program officials reported various reasons for the changes in their 
planned schedule estimates. For example, officials for the Navy’s CAC2S 
program attributed the schedule slippages to the addition of new 
requirements and program restructuring. Officials for the Air Force’s AOC- 
WS Inc 10.2 program reported that the 3 years of schedule slippages 
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were due to a combination of test problems and system maturity issues. 
In addition, officials for the Air Force’s JMS Inc 2 system cited a larger 
than expected number of discrepancies in testing that had caused 
schedule delays totaling 2 years and 10 months. In turn, the program 
delayed completion of its Milestone C and Full Deployment Decision by 
more than 1 year beyond the original estimate. 

Four MAIS programs experienced no change to their planned schedule 
estimates—the DAI Inc 2, DHMSM, IPPS-A Inc 2, and the ISPAN Inc 4 
programs. 


Nine Selected MAIS 
Programs Had Met 
Technical Performance 
Targets, while Others 
Were at Various Stages 
of Progress 


Among other information, DOD uses key performance parameters as a 
metric to report on each program’s progress toward meeting the system 
technical performance targets. This information includes a description of 
the performance characteristics, the objective and threshold value for 
each target and, importantly, whether the target has been met in 
demonstrating performance. For example, one program identified 
“network readiness and exchange of information” as a technical 
performance target. The program also established an objective value for 
this target as successfully processing 98 percent of the information into 
the system, and established an threshold value for this target as 
successfully processing 95 percent of the information into the system. 

Among the 18 selected MAIS programs we reviewed that had established 
technical performance targets, nine had met all of their defined targets. 

As an example of technical performance targets that had been met, the 
Army’s GCSS Inc 1 program reported in the 2016 Major Automated 
Information System Annual Report that both of its technical performance 
targets had been met. These targets related to the readiness of the 
system to fully support operational activities and information exchanges in 
the architecture (referred to as Net Ready) and compliance with the 
operational activities that represent the system’s mission critical functions 
(referred to as Mission Critical Functions). 

As another example, the Defense Health Agency’s iEHR Inc 1 program 
reported that all 3 performance targets had been met. These targets 
related to ensuring that end user devices display the same information, 
context, and display of the patient, expediting user access to the 
applications, and ability of users to move from one device to another. 

Further, of the 18 MAIS programs, four programs—the Air Force’s BITI- 
Wired and DEAMS Inc 1 programs, Navy’s CANES and Defense 
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Information Systems Agency’s Teleport Gen 3 programs—had partially 
met their technical performance targets. For example, BITI-Wired had 
partially implemented all 4 of its performance targets—related to 
readiness of the network, availability of redundant features such as dual 
power supplies and spares, reliability of systems, and the installation of 
support equipment at all 178 sites. According to program officials, support 
equipment had been installed at 167 of the 178 total sites and the 
program expects all 4 performance targets to be met in September 2017, 
the planned full deployment date. As another example, the Air Force’s 
DEAMS Inc 1 program officials reported that 6 of 9 targets had been met. 
For the 3 performance targets that had not been met, officials reported 
that processing issues had prevented them from meeting these targets, 
which included establishing accurate fund balances, reporting in a timely 
manner, and ensuring readiness of the network. As of June 2016, 
program officials reported they were working to address these issues. 

Finally, five programs—IPPS-A Inc 2, AOC-WS Inc 10.2, ISPAN Inc 4, 
JMS Inc 2, and DHMSM—were in the early stages of the system 
development and had not begun testing. As a result, it is too early to 
determine the status of performance targets because information on the 
status of technical performance is not yet available. 
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All Five Selected 
MAIS Programs Had 
Fully or Partially 
Implemented Leading 
Practices for 
Managing Risk, 
Requirements, and 
Systems Testing and 
Integration 


According to the Software Engineering Institute’s Capability Maturity 
Model® Integration for Acquisition (CMMI- ACQ), 15 leading practices for 
managing an IT program include risk management, and requirements 
management. Effective requirements management includes establishing 
an agreed-upon set of requirements and ensuring traceability between 
requirements and work products. Further, changes to requirements 
should be managed in collaboration with stakeholders for the acquisition 
of systems within their established cost, schedule, and delivery of the 
expected product. Likewise, an effective risk management process 
includes identifying problems before they occur, so that risk-handling 
activities may be planned and invoked, as needed, across the life of the 
project in order to mitigate the potential for adverse impacts. Further, 
regarding systems testing and integration, other leading practices from 
the Institute of Electrical and Electronics Engineers, Incorporated (IEEE) 
and DOD policy guidance 16 (DOD Instruction 5000.02, Defense 
Acquisition Guidebook, and Test & Evaluation Management Guide) state 
that roles and responsibilities should be established and test-related 
plans, schedules, and reports should be developed to better manage test 
and integration activities. 


All five MAIS programs that we selected for an in-depth review had fully 
or partially implemented leading practices for managing risk, 
requirements, and systems testing and integration. Table 3 shows the 
extent to which these programs had implemented the practices for 
requirements management, risk management, and systems testing and 
integration management. We assessed one program as “not applicable” 
because it was initiated after a limited production decision point (post- 
Milestone C) and, therefore, had not implemented certain practices. 


15 Software Engineering Institute, Capability Maturity Model® Integration for Acquisition 
(CMMI-ACQ), Version 1.3 (Pittsburgh, Pa.: November 2010; November 2010). 

16 Department of Defense, Operation of the Defense Acquisition System, DODI 5000.02 
(January 7, 2015); Department of Defense, Defense Acquisition Guidebook ; Department 
of Defense, Test and Evaluation Management Guide 6th Edition (December 2012). 
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Table 3: Extent to Which Selected Major Automated Information Systems Programs Applied Leading Practices for 
Requirements and Risk Management and Systems Testing and Integration 


Leading Practice 

Air Force AOC- 
WS Inc 10.2 

Air Force BITI 
Wired 

Air Force JMS 
Inc 2 

Army GCSS-A 
Inc 1 

Army IPPS-A 
Inc 2 

Requirements management 

1. Develop an understanding with the 
requirements providers on the 
meaning of the requirements 

• 

• 

• 

9 

• 

2. Obtain commitment to 
requirements from project 
participants 

• 

N/A 

• 

9 

• 

3. Manage changes to requirements 
as they evolve during the project 

9 

• 

• 

9 

• 

4. Maintain bidirectional traceability 
among requirements and work 
products 

• 

N/A 

• 

9 

• 

5. Ensure that project plans and work 
products remain aligned with 
requirements 

• 

N/A 

• 

9 

• 

Risk management 

1. Determine risk sources and 
categories used to examine and 
oversee changes that impact the 
project. 

• 

• 

• 

• 

• 

2. Define parameters used to analyze 
and categorize risks and to control 
the risk management effort. 

• 

• 

• 

9 

• 

3. Establish and maintain the 
strategy to be used for risk 
management. 

9 

• 

• 

• 

• 

4. Identify and document risks that 
could negatively affect work 
efforts. 

9 

• 

• 

• 

• 

5. Evaluate and categorize each 
identified risk using defined risk 
categories and parameters, and 
determine its relative priority. 

• 

• 

• 

• 

• 

6. Develop a risk mitigation plan in 
accordance with the risk 
management strategy. 

9 

• 

• 

• 

• 

7. Monitor the status of each risk 
periodically and implement the risk 
mitigation plan as appropriate. 

• 

• 

• 

9 

• 

Systems Testing & Integration 
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Leading Practice 

Air Force AOC- 
WS Inc 10.2 

Air Force BITI 
Wired 

Air Force JMS 
Inc 2 

Army GCSS-A 
Inc 1 

Army IPPS-A 
Inc 2 

1. Establish roles and responsibilities 
to manage testing/integration 
activities. 

• 

• 


• 

• 

2. Formulate test related plans, 
schedules, and reports. 

• 

• 

• 

• 

• 


Source: GAO analysis of data provided by DOD officials | GAO-17-322 


Legend: AOC-WS Inc 10.2 = Air and Space Operations Center-Weapon System Increment 10.2; BITI 
Wired = Base Information Transport Infrastructure Wired; JMS Inc 2 = Joint Space Operations 
Center, Mission System Increment 2; GCSS-A Inc 1 = Global Combat Support System-Army 
Increment 1; IPPS-A Inc 2 = Integrated Personnel and Pay System—Army Increment 2. 

• -Fully implemented indicates that the agency provided evidence that it had addressed all of the sub 
practices in a category 

o - Partially implemented indicates that the agency provided evidence that it had addressed at least 
one, but not all, of the sub practices in a category 

N/A - Not applicable indicates that the agency did not complete this practice because the program 
was initiated post-Milestone C. 


Two of the Five MAIS 
Programs Had Not Fully 
Implemented All Practices 
for Managing 
Requirements 


Leading requirements management practices include establishing an 
agreed-upon set of requirements, ensuring traceability between 
requirements and work products, and managing any changes to the 
requirements in collaboration with stakeholders. Such leading practices 
help organizations to better manage the design, development, and 
delivery of systems within established cost and schedule time frames. 
These practices include 


• developing an understanding with the requirements providers on the 
meaning of the requirements, 

• obtaining commitment to requirements from project participants, 

• managing changes to requirements as they evolve during the project, 

• maintaining bidirectional traceability among requirements and work, 
and 

• ensuring that project plans and work products remain aligned with 
requirements. 

Three of the five programs had fully implemented the requirements 
management practices, while the other two had partially implemented 
some practices. 
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Air Force— Air and Space Operations Center-Weapon System 
Increment 10.2 (AOC-WS Inc 10.2) 

The Air Force had fully implemented four, and partially implemented one, 
of the five requirements management practices for the AOC WS Inc 10.2 
program. For example, the program developed an understanding with 
requirements providers on the meaning of the requirements and 
established a requirements working group that oversees the development 
and management of program requirements. In addition, the AOC-WS Inc 
10.2 program obtained commitment to requirements from project 
participants through the DOD configuration control board process, in 
which requirements were reviewed and, upon acceptance, approved, 
documented, and tracked. The program also managed changes to 
requirements as they evolved during the project; however, although 
changes to requirements were documented and maintained, the program 
did not provide evidence of its rationale for the changes. The AOC-WS 
Inc 10.2 program has since completed its acquisition phase and, 
therefore, providing a rationale for changes made during the requirements 
development phase would not impact the program moving forward. 

Air Force— Base Information Transport Infrastructure Wired (BITI 
Wired) 

The Air Force had fully implemented the two applicable requirements 
management practices for the BITI Wired program. Specifically, the 
program had developed an understanding with requirements providers on 
the meaning of the requirements and had managed changes to 
requirements as they evolved during the project. The BITI Wired 
requirements management plan established the criteria for the evaluation 
and acceptance of the requirements. 

On the other hand, three of the five requirements—obtain commitment to 
requirements from project participants; maintain bidirectional traceability 
among requirements and work products; and ensure that project plans 
and work products remain aligned with requirements—were not 
applicable for the BITI Wired Program. This was due to the Air Force 
beginning the program after Milestone C. 

Air Force—Joint Space Operations Center, Mission System 
Increment 2 (JMS Inc 2) 

The Air Force had fully implemented the five requirements management 
practices for the JMS Inc 2 program. Specifically, the program had 
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developed an understanding with requirements providers on the meaning 
of the requirements. For example, the program’s Requirements 
Engineering and Management Plan established criteria for distinguishing 
appropriate requirements and for evaluating and accepting requirements. 
In addition, changes to requirements were managed as they evolved 
during the project. For example, requirements change requests had been 
sent through the Lead Command, which validated them and sent them to 
the Program Management Office for approval. Lastly, the program 
maintained requirements traceability from source requirements to lower- 
level requirements to ensure that all source requirements had been 
completely addressed. Specifically, the program had established 
processes for managing the program’s Requirements Traceability Matrix 
database, which included activities to enter approved requirements, 
sequence traceability changes, update development, test status, and 
report data. 

Army— Global Combat Support System-Army Increment 1 (GCSS-A 
Inc 1) 

The Army had partially implemented all five requirements management 
practices for the GCSS-A Inc 1 program. Officials reported that, although 
the program had progressed to the sustainment phase, and its 
requirements activities had been completed since 2014, the Configuration 
Steering Board review process had been used to manage, oversee, and 
approve requirements as they evolved during the project. The program 
had developed a Software Development Overview Schedule that 
provided a general overview of their software development life cycle 
process. 

Nevertheless, an approved and signed Requirements Management Plan 
at program initiation had not been implemented. Such a plan would have 
been important to ensuring an understanding with the requirements 
providers of the meaning of the requirements. Further, the program’s 
Release Management Plan included a section on the requirements 
process and references to a central information repository in the 
Requirements Traceability database. However, the requirements 
traceability matrix provided did not contain all relevant information such 
as “priority.” These important management tools could have provided the 
program with the discipline, structure, and oversight typically associated 
with successful organizations that apply this practice. However, since the 
GCSS-A Inc 1 program has completed its acquisition phase, fully 
implementing these practices now would not impact the program moving 
forward. 
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Army—Integrated Personnel and Pay System—Army Increment 2 
(IPPS-A Inc 2) 

The Army had fully implemented the five requirements management 
practices for the IPPS-A Inc 2 program. Specifically, the program had 
developed an understanding with requirements providers on the meaning 
of the requirements and had managed changes to requirements as they 
evolved during the project. For example, the program’s Requirements 
Management Plan described the criteria and process through which 
requirements are evaluated and accepted. In addition, the IPPS-A 
Requirements/Configuration Control Board Charter requires that all 
changes be vetted through senior governance boards, such as the 
Council of Colonels and General Officer’s Steering Committee. Further, 
the program maintained bidirectional traceability of requirements. 
Specifically, a history for all requirements changes was maintained within 
a repository for requirements that were managed by the Application 
Lifecycle Management Tool. 


Two of the Five MAIS 
Programs Had Not Fully 
Implemented All Practices 
for Managing Risk 


According to leading practices, an effective risk management process 
identifies potential problems before they occur, so that risk-handling 
activities may be planned and invoked, as needed, across the life of the 
project in order to mitigate the potential for adverse impacts. These 
practices include 


• determining risk sources and categories; 

• defining parameters used to analyze and categorize risks and to 
control the risk management effort; 

• establishing and maintaining the strategy to be used for risk 
management; 

• identifying and documenting risks; 

• evaluating and categorizing each identified risk using defined risk 
categories and parameters, and determining its relative priority; 

• developing a risk mitigation plan in accordance with the risk 
management strategy; and 

• monitoring the status of each risk periodically and implementing the 
risk mitigation plan as appropriate. 

Three of the five programs had fully implemented the risk management 

practices, while two had partially implemented some practices. 
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Air Force—AOC-WS Inc 10.2 


The Air Force had fully implemented four, and partially implemented 
three, of the seven risk management practices for theAOC WS Inc 10.2 
program. For example, the program had determined risk sources and 
categories used to examine and oversee changes that impact the project, 
as described in the program’s Risk Management Plan. The program also 
had defined parameters used to analyze and categorize risks and to 
control the risk management effort. Specifically, the Risk Management 
Plan evaluated and assigned values to each risk by calculating the 
probability and consequence of a risk, as well as its likelihood. 

The AOC-WS Inc 10.2 program had partially implemented the key 
management practice to identify risks that could negatively affect work 
efforts. For example, although the program identified risk in specific risk 
categories, it did not identify risks that were outside the normal scope of 
the project. 

In addition, while the program had developed criteria for evaluating and 
qualifying risk likelihood and severity levels, it did not develop an overall 
risk mitigation plan in accordance with the risk management strategy. 

This occurred because the program underwent a critical change and, 
according to program officials, this critical change altered the 
management of the program due to its complexity. Until an overall risk 
mitigation plan is developed, the program will not be able to effectively 
manage risk mitigation and contingency planning activities. 

Air Force—BITI Wired 

The Air Force had fully implemented all seven of the risk management 
practices for the BITI Wired program. Specifically, the program had 
defined parameters used to analyze and categorize risks and to control 
the risk management effort. For example, the program evaluated and 
assigned values to each risk by calculating the probability and 
consequence of a risk as well as its likelihood and consequence. The 
program had also identified and documented risks that could negatively 
affect work efforts. For example, the Risk Management Plan established 
risk conditions, thresholds, and probabilities of occurrence. In addition, 
the BITI Wired program monitored the status of each risk periodically and 
implemented risk mitigation plans as appropriate through the use of a risk 
management tool and risk board. 
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Air Force—JMS Inc 2 


The Air Force had fully implemented all seven of the risk management 
practices for the JMS Inc 2 program. Specifically, the program had 
defined parameters used to analyze and categorize risks and to control 
the risk management effort. For example, as detailed in the program’s 
Risk and Opportunity Management Plan, risk ratings, including likelihood 
and consequence of occurrence, were established through the risk 
matrix; this provided a consistent, standardized method for risk analysis. 
In addition, defined thresholds were used to prioritize risks and trigger 
management action. Further, the program had developed risk 
management plans establishing a strategy used for risk management. 
Specifically, risks were monitored and risk status was reviewed in the 
issue database as well as through meetings used to report on 
programmatic metrics. Standard operating procedures had also been 
developed to manage risk, including one to manage risk identification, 
analysis, mitigation, and tracking. 

Army—GCSS-A Inc 1 

The Army had fully implemented five, and partially implemented two, of 
the risk management practices for the GCSS-A Inc 1 program. 
Specifically, the program had a Risk and Opportunity Management Plan 
that identified risk sources and categories. Further, in order to control the 
risk management effort, risks were categorized and grouped according to 
defined risk categories in the program’s Risk Management Plan. Further, 
identified risks were evaluated using defined risk parameters. For 
example, the Risk Register identified risk consequence and probability. 

On the other hand, the program did not define thresholds or bounds for 
mitigation or management action. According to leading practices, the risk 
management effort can be prioritized, controlled, and managed better by 
defining thresholds and categorizing risks for mitigation/management 
action. Although program officials stated that they were in the process of 
continuing to optimize their risk management efforts, and were updating 
their standard operating procedures, this key leading practice has not yet 
been implemented. Until the program has, and is guided by, updated 
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procedures, it will not be positioned to ensure that risks can be effectively 
prioritized, controlled, and managed. 17 

Army—IPPS-A Inc 2 

The Army had fully implemented all seven of the risk management 
practices for the IPPS-A Inc 2 program. Specifically, the program’s Risk 
Management Plan provided guidelines that determined risk sources and 
categories. These risks were categorized in the Risk Management Tool 
based on the “Risk Taxonomy” found in the Risk Management Plan. 
Specifically, the risks were assigned to three main categories—technical, 
programmatic, and external risks. The program also ensured potential 
risks were identified and documented. For example, the Defense 
Information Systems Agency, which hosts the two IPPS-A Defense 
Enterprise Computing Centers (in St. Louis, MO, and Ogden, UT), 
conducted assessments to ensure the sites were in compliance with 
environmental controls. Further, the program developed risk mitigation 
plans that identified the person or group responsible for addressing each 
risk through the Risk Register. 


Four of the Five MAIS 
Programs Had Fully 
Implemented All Practices 
for Managing Systems 
Testing and Integration 

• establishing roles and responsibilities to manage testing and 
integration activities and 

• developing test-related plans, schedules, and reporting. 

Four of the five programs had fully implemented practices for systems 
testing and integration. 


According to leading practices for managing systems testing and 
integration activities, roles and responsibilities should be established and 
test-related plans, schedules, and reports should be developed. Doing so 
provides a baseline by which to measure progress so that appropriate 
corrective actions can be taken when the program’s performance 
deviates from the plans. These practices include 


17 Subsequent to sending a draft of this report to DOD for comment in February 2015, the 
Army’s GCSS-A Inc 1 program manager amended their Risk Management Plan for Project 
Manager Army Enterprise Systems Integration Program on March 1,2017. This 
acquisition decision brought the program back into compliance with this leading practice. 
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Air Force—AOC-WS Inc 10.2 


The Air Force had fully implemented both systems testing and integration 
practices for the AOC-WS Inc 10.2 program. The program established 
roles and responsibilities to manage testing/integration activities and 
formulated test related plans, schedules, and reports. Specifically, the 
program’s Integrated Test Team Charter and the Test and Evaluation 
Master Plan (TEMP) established organizational and individual roles and 
responsibilities fortesting and integration. Further, the program 
developed a test plan that included a test and evaluation strategy and 
schedule. 

Air Force—BITI Wired 

The Air Force had fully implemented both systems testing and integration 
practices for the BITI Wired program. In this regard, the program had 
established testing roles and responsibilities. The program also 
formulated all applicable test related plans, schedules, and reports. For 
example, the BITI Wired Program developed testing plans and schedules 
for each base installation site. 

Air Force—JMS Inc 2 

The Air Force had fully implemented one, and partially implemented one, 
of the systems testing and integration practices for the JMS Inc 2 
program. For example, the program had developed a Test and Evaluation 
Master Plan that provided the framework and strategy for planning and 
executing its testing and evaluation activities. Although the program had 
established roles and responsibilities to manage testing and integration 
activities, a chief developmental tester to oversee testing activities had 
not been identified. Program officials stated that it had been difficult to fill 
this position due to the lack of qualified personnel. Until this position is 
filled, it may be difficult to manage and mitigate risks during development, 
verify that products are compliant with operational requirements, and 
inform decision makers throughout the program’s life cycle. 

Army—GCSS-A Inc 1 

The Army had fully implemented both systems testing and integration 
practices for the GCSS-A Inc 1 program. In this regard, the program had 
established and documented roles and responsibilities for all testing and 
evaluation activities, including identifying a chief developmental tester. In 
addition, a GCSS-A Inc 1 Test and Evaluation Master Plan had been 
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developed, which described the total testing and evaluation planning from 
component development through operational testing and evaluation. The 
program also had developed test plans, such as the Capstone Test Plan 
and Systems Engineering Plan, which further detailed the program’s 
testing and evaluation strategy. 

Army—IPPS-A Inc 2 

The Army had fully implemented both systems testing and integration 
practices for the IPPS-A Inc 2 program. In this regard, the program had 
established testing roles and responsibilities and formulated test plans, 
schedules, and reports, where applicable. Specifically, the Increment II 
Test and Evaluation Working Integrated Product Team Charter was 
established to provide guidance in overseeing testing activities. In 
addition, a chief developmental tester had been appointed to serve as the 
focal point for all testing and evaluation efforts. Further, the IPPS-A Inc 2 
program developed a Test and Evaluation Master Plan, which described 
the overall testing and evaluation plans from system development through 
operational testing and evaluation for all releases. 


Conclusions 


MAIS programs are important for sustaining DOD’s key operations in 
communications, business, and command and control, and provide the 
department with access to information to organize, plan, direct, and 
monitor mission operations. Therefore, it is vital that these programs 
follow leading practices to ensure success. While a number of leading 
practices for risk, requirements, and systems testing and integration had 
been fully or partially implemented by five programs that we reviewed in- 
depth—the Air Force’s AOC WS Inc 10.2, BITI Wired and JMS Inc 2 
programs; and the Army’s GCSS-A Inc 1 and IPPS-A Inc 2 programs— 
three programs lacked practices to better manage the development of 
systems. The program office for GCSS-A Inc 1 did not have standard 
operating procedures for defining thresholds or bounds to manage risks, 
thus putting at risk the program’s ability to effectively prioritize, control, 
and manage for mitigation and management actions. In addition, the 
program office for AOC-WS Inc 10.2 did not have an overall Risk 
Mitigation Plan to guide the implementation of individual risk mitigation 
and contingency planning activities. An overall risk plan is necessary to 
help ensure that individual efforts are managed in a cohesive and 
repeatable way. Further, the program office for JMS Inc 2 had not 
assigned a chief developmental tester, an important role to ensure that 
risks are managed and mitigated effectively throughout testing and 
integration. Unless this position is filled, it will be difficult for the program 
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to effectively manage risks and verify compliance with certain operational 
requirements throughout the acquisition. 


Recommendations for To help improve the management of DOD's MAIS programs, we are 

recommending that the Secretary of Defense take the following three 

Executive Action actions: 

• direct the Secretary of the Army to direct the program manager for 
GCSS-A Inc 1 to establish standard operating procedures for 
managing risks that include guidance for establishing thresholds and 
bounds for key risk areas. 

• direct the Secretary of the Air Force to direct the program manager for 
AOC-WS Inc 10.2 to develop an overall risk mitigation plan to guide 
the implementation of individual risk mitigation and contingency plan 
activities. 

• direct the Secretary of the Air Force to direct the program manager for 
JMS to appoint a chief developmental tester to oversee systems 
testing and integration activities. 


DOD provided written comments on a draft of this report, which are 
reproduced in appendix III. In its comments, the department agreed with 
all three of our recommendations. Further, the department provided 
evidence that it had fully addressed one of the recommendations and it 
described plans and actions for addressing the remaining two 
recommendations. 

Regarding our recommendation that the GCSS-A Inc 1 program establish 
standard operating procedures for managing risks, DOD provided 
evidence that the Army had updated its risk management plan. 
Specifically, the Army’s GCSS-A Inc 1 program manager amended the 
Risk Management Plan for Project Manager Army Enterprise Systems 
Integration Program on March 1, 2017. The plan now describes how risk 
prioritization occurs; it requires that risk owners work with the risk 
management officer to define appropriate trigger conditions in order to 
ensure a mitigation plan can be initiated at the appropriate point. By 
taking these actions to implement this recommendation, the program 
should be in a better position to more effectively manage risks. 

DOD also concurred with our recommendation to develop an overall risk 
mitigation plan to guide the implementation of individual risk mitigation 
and contingency planning activities on the AOC WS Inc 10.2 program. 


Comments from the 
Department of 
Defense 
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Toward this end, the Air Force reported that it plans to restore the 
program’s risk management practices to include weekly meetings to 
review program risks and publish a risk management plan in April 2017. 
Once implemented, these actions should help the Air Force to better 
ensure that individual risk mitigation and contingency plan activities are 
conducted in a consistent and repeatable way. 

Further, DOD concurred with our recommendation that JMS appoint a 
chief developmental tester to oversee systems testing and integration 
activities. In this regard, the Air Force reported that it had identified a 
qualified candidate to fill the Chief Developmental Tester position. 
According to the Air Force, the program expects the new test chief to be 
in place in July 2017. Once the position is filled, the program should be 
better able to oversee important systems testing and integration activities. 


We are sending copies of this report to the appropriate congressional 
committees; the Secretary of Defense; the Secretary of the Army; the 
Secretary of the Air Force; and other interested parties. This report also is 
available at no charge on the GAO website at http://www.gao.gov. 

Should you or your staffs have any questions on information discussed in 
this report, please contact me at (202) 512-4456 or harrisc@gao.gov. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. GAO staff who made 
major contributions to this report are listed in appendix IV. 



Carol C. Harris 
Director 

Information Technology Acquisition Management Issues 
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Methodology 


The National Defense Authorization Act for Fiscal Year 2012 mandated 
that we select, assess, and report on Department of Defense’s (DOD) 
major automated information system (MAIS) programs annually through 
March 2018. 1 This report is the fifth in our series of annual assessments. 
Our objectives were to: (1) describe the extent to which selected MAIS 
programs have changed their planned cost and schedule estimates and 
met technical performance targets and (2) assess the extent to which 
selected MAIS programs have used leading IT acquisition practices, 
including requirements and risk management, and systems testing and 
integration. 

To address the first objective, we used DOD’s official list of 35 MAIS 
programs as of May 1, 2016, to establish a basis for selecting programs. 
We selected 18 of 35 MAIS programs based on our criteria that programs 
must be unclassified and have a first acquisition program baseline that 
could be used as a reference point for evaluating cost, schedule, and 
technical performance characteristics. 

For each selected program, we asked relevant DOD officials to complete 
a data collection instrument with questions on estimated cost, schedule, 
and technical performance goals, including the latest program status in 
meeting those estimated goals. Using the responses from the data 
collection instrument, we then compared each program’s first acquisition 
program baseline cost estimate to the latest estimate to determine the 
extent to which planned program costs had changed. Similarly, to 
determine the extent to which these programs had changed their planned 
schedule estimates, we compared each program’s first acquisition 
program baseline schedule estimate to the latest schedule. 

To determine whether the selected programs met their performance 
targets, we reviewed the most recent program management reports and 
examined each program’s self-identified system performance targets and 
the status of their progress. We also reviewed additional information on 
each program’s cost, schedule, and performance, including program 
documentation, such as DOD’s MAIS annual and quarterly reports, 
acquisition program baselines, system test reports, and our prior reports. 
We then aggregated and summarized the results of these analyses 
across the programs, and developed individual profiles for each program. 


Vub. L. No. 112-81 § 1078 (Dec. 31,2011). 
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To address the second objective, we started with the list of 18 MAIS 
programs that were selected specifically for the first objective. From this 
list, we identified programs for a more in-depth review based on two 
criteria: the programs selected must represent at least two military 
services, and the programs must have not been evaluated by us within 
the last year. 2 Using this criteria, we identified 5 programs from two 
military services: the Army’s Global Combat Supply System Increment 1 
(GCSS-A Inc 1) and Integrated Personnel and Pay System Increment 2 
(IPPS-A Inc 2); and the Air Force’s Air and Space Operations Center- 
Weapon System Increment 10.2 (AOC-WS Inc 10.2), Base Information 
Transport Infrastructure Wired (BITI Wired), and Joint Space Operations 
Center, Mission System Increment 2 (JMS Inc 2). 

Based on prior work evaluating MAIS programs, we identified three 
management practice areas—requirements management, risk 
management, and systems testing and integration that are applicable to 
these programs. For evaluating how the selected five programs applied 
practices in the risk and requirements management areas, we identified 
key practices related to these areas from the Software Engineering 
Institute’s Capability Maturity Model® Integration for Acquisition (CMMI- 
ACQ). 3 

Specifically, for requirements management, we evaluated each of the five 
programs against the following 5 practices: 

• develop an understanding with the requirements providers of the 
meaning of the requirements, 

• obtain commitment to requirements from project participants, 

• manage changes to requirements as they evolve during the project, 

• maintain bidirectional traceability among requirements and work, and 

• ensure project plans and work products remain aligned with 
requirements. 

In doing so, we analyzed program requirements documentation against 
the practices. This documentation included requirements management 
plans, traceability matrices, requirements change forms, technical 


2 GAO-16-336 

3 Software Engineering Institute, Capability Maturity Model® Integration for Acquisition 
(CMMI-ACQ), Version 1.3 (Pittsburgh, Pa.: November 2010). 
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performance assessments, and requirements board meeting minutes. We 
also interviewed program officials to obtain additional information about 
their requirements management practices. 

For the risk management area, we evaluated each of the five programs 
against the following 7 practices: 

• determine risk sources and categories; 

• define parameters used to analyze and categorize risks and to control 
the risk management effort; 

• establish and maintain the strategy to be used for risk management; 

• identify and document risks; 

• evaluate and categorize each identified risk using defined risk 
categories and parameters, and determine its relative priority; 

• develop a risk mitigation plan in accordance with the risk management 
strategy; and 

• monitor the status of each risk periodically and implement the risk 
mitigation plan as appropriate. 

In doing so, we analyzed each program’s key documents, such as risk 
management plans, risk charts, and other artifacts, to determine the 
extent to which these practices had been applied. We also interviewed 
program officials to obtain additional information about their risk 
management practices. 

For the systems testing and integration area, we evaluated each of the 
five programs against the following 2 practices: 

• establish roles and responsibilities to manage testing and integration 
activities and 

• develop test-related plans, schedules, and reporting. 

In this regard, we evaluated the test manager’s roles and responsibilities, 
as well as test plans, schedules, and test reports to determine the extent 
that these practices had been applied. We also conducted interviews with 
program officials to obtain additional information about their systems test 
and integration practices. 

Further, in determining the extent to which each of the five MAIS 
programs had implemented the three practice areas—requirements 
management, risk management, and systems testing and integration—we 
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rated a practice as “fully implemented” when all of the practices for the 
area had been implemented. Similarly, when a program had implemented 
at least one of the practices within the area, we rated the practice as 
“partially implemented” to indicate the practice had been partially 
implemented. For a MAIS program that did not conduct activities for any 
of the practices because the program had been initiated after a limited 
production decision point (post-Milestone C), we assessed the practice as 
“not applicable” because such programs do not apply practices to 
develop, manage requirements, or test the systems. 

To assess the reliability of the data that we used to support the findings in 
this report, we corroborated program office responses with relevant 
program documentation and interviews with agency officials. We found no 
data reliability issues and determined that the data used in this report 
were sufficiently reliable for our reporting purposes. We have also made 
appropriate attribution indicating the sources of the data. 

We conducted this performance audit from April 2016 to March 2017 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 
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Programs 


This section contains profiles of the 18 selected MAIS programs that we 
analyzed to determine whether they had changed their planned cost and 
schedule estimates and met performance measures. Each profile 
presents data on the program’s purpose and status, its latest cost and 
schedule estimates compared to the first acquisition program baseline 
and system performance data. The first page of each two-page profile 
provides (1) a description of the program’s purpose, (2) essential details, 
such as the responsible DOD component, program owner, prime 
contractor, the total number of active contractors supporting the program; 
and (3) program costs (in then-year dollars), 1 comparing the program’s 
latest life-cycle cost estimate (separated into acquisition and operations 
and maintenance costs) to its first acquisition program baseline 
(subsequent acquisition program baselines that may have been 
established are not identified). 

The profile also provides information on the program’s status, costs, 
schedule, and performance. 

• In the program status section, we discuss recent milestones achieved 
and acquisition events. 

• In the cost section, we identify the extent to which the program’s life- 
cycle cost estimate has changed from its first acquisition program 
baseline, as well as any notable causes for any changes. 

• In the schedule section, we discuss the extent to which the program’s 
schedule has changed from its first acquisition program baseline and 
causes for significant changes. 

In the performance section, we identify the extent to which each program 
has met its established measures. These performance ratings represent 
point-in-time assessments as reported by the program. System 
performance targets are rated as “met” when the program fully met all of 
its key performance parameters. System performance was rated as “did 
not fully meet” when a program did not pass system testing. System 
performance was rated as “not yet applicable” when the system had not 
yet been tested because the program was not yet at the appropriate 
acquisition phase. 


^hen-year dollars are those dollars that include the effects of inflation or escalation 
and/or reflect the price levels expected to prevail during the year at issue. 
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Air Force—Air and Space 
Operations Center- 
Weapon System 
Increment 10.2 (AOC-WS 
Inc 10.2) 


Program description 

The AOC-WS Inc 10.2 program is expected to provide system hardware, 
software, technical documents, and technology refresh to ensure it 
remains a viable weapons system. The program will enable personnel at 
selected air and space operations centers to better plan, execute, and 
assess theater-wide air and space operations. It will replace the currently 
fielded AOC 10.1 system and provide additional capabilities, such as 
dynamic planning and execution; data management; information 
assurance; predictive battlespace awareness; and airspace management. 


Program essentials (as of October 2016) 

DOD component: Air Force 

Program owner: Air Combat Command 

Prime contractor: Northrop Grumman, Mission Systems 

Total number of contractors (as of May 2016): 10 


Table 4: Air and Space Operations Center-Weapon System Increment 10.2 Program 
costs 

(then-year dollars in millions) 


First acquisition 
program baseline 
October 2013 

Latest 
estimate 
June 2016 

Acquisition cost estimate 

Research, development, test, and 
evaluation 

$374.7 

$604.2 

Procurement 

88.0 

101.8 

Total acquisition cost 

462.7 

706.0 

Life-cycle cost estimate 

Operations and support 

5,122.5 

4,936.0 

Total life-cycle cost estimate 

$5,585.2 

$5,642.0 


Source: DOD data | GAO-17-322 
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Program status 

As of June 2016, the AOC-WS Inc 10.2 program was in the engineering 
and manufacturing development phase of acquisition in the Defense 
Acquisition Management System. In March 2016, the program declared a 
critical change because of significant cost and schedule growth due to 
schedule and technical difficulties in reaching Milestone C, which is the 
key juncture for approval to move into limited production. According to the 
program office, major factors contributing to the cost and schedule delays 
were the prime contractor (1) underestimating the software development 
effort; (2) insufficiently allocating manpower; and (3) underestimating the 
end-to-end testing capability. 

Exceeded planned cost estimate 

As of June 2016, the AOC-WS Inc 10.2 program’s total life-cycle cost was 
$5,642.0, an increase of $56.8 million (1 percent) from the original 
estimate of $5,585.2. The estimated total acquisition cost is $706.0 
million, an increase of $243.3 million (53 percent) from the original 
estimate of $462.7 million. 

Exceeded planned schedule estimate 

The AOC-WS Inc 10.2 program experienced significant schedule delays 
since establishing the original schedule in 2007. Specifically, Milestone C 
was estimated to be completed on July 2015, but, as of September 2016, 
the program had not reached the milestone. According to program 
officials, the program was undergoing a critical change that will result in a 
3-year schedule delay. 

Not yet applicable 

The AOC-WS Inc 10.2 program was still early in development and system 
performance data was not available at the time of our review. 
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Air Force—Base 
Information Transport 
Infrastructure Wired 
(BITI Wired) 


Program description 

The BITI Wired program is to upgrade the enterprise network and provide 
the core cyber infrastructure. The upgrade includes wired and wireless 
systems, and network cables and servers for more than 150 active duty, 
Air Force Reserve, and Air National Guard bases. The program is 
expected to update the fixed local area network and all necessary 
information transport infrastructure to support current and future 
communications needs of the Air Force and Joint Command warfighter. 
This includes enabling base level data, voice, video, imagery, sensor, and 
telemetry requirements. 

Program essentials (as of October 2016) 

DOD component: Air Force 

Program owner: Air Force Space Command (AFSPC) 

Prime contractors: General Dynamics, Harris Corp, Lockheed Martin, 
Telos Corp, Three Wire Systems, and Red River for the procurement 
contracts and Affigent and ThunderCat for the operations and support 
contracts 

Total number of contractors (as of May 2016): 8 


Table 5: Base Information Transport Infrastructure Wired Program costs 

(then-year dollars in millions) 


First acquisition 
program baseline 

June 2010 

Latest 
estimate 
July 2016 

Acquisition cost estimate 

Research, development, test, and 
evaluation 

$0.0 

$0.0 

Procurement 

1,417.7 

1,177.2 

Total acquisition cost 

1,417.7 

$1,177.2 

Life-cycle cost estimate 

Operations and support 

2,372.2 

1,146.5 

Total life-cycle cost estimate 

$3,789.9 

$2,323.7 


Source: DOD data | GAO-17-322 
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Program status 

The BITI-Wired program achieved a full deployment decision on February 
18, 2014, and expects to reach full deployment in September 2017. The 
program completed 167 out of 178 turnkey network upgrades and 
expects to remain within the cost, schedule, and performance thresholds. 

Decreased cost estimate 

As of July 2016, the BITI Wired program’s total life-cycle cost estimate 
was $2,323.7 million, a decrease of $1,466.2 million (-39 percent) from 
the original estimate of $3,789.9 million. The decrease was due to more 
accurate cost estimates being provided by vendors. Specifically, initial 
estimates included large risk factors because, at the time, the government 
could not provide vendors a definitive list of items. As a result, vendors’ 
earlier estimates reflected the potential for risks in their estimates. The 
new list of refined equipment resulted in reduced costs. In addition, while 
officials cited insufficient human resources as a potential risk to cost as 
the program moves forward, they fully expect to stay within the cost 
estimates. 

Met planned schedule estimate 

The BITI Wired program’s full deployment decision was 2 months’ behind 
schedule. However, program officials report they are on track to meet full 
deployment in September 2017. 

Did not fully meet all of the system performance targets 

The BITI-Wired program had partially implemented all 4 of performance 
targets—net ready, availability, support, and reliability. According to 
program officials, installation of equipment had been installed at 167 of 
the 178 sites. The program expects to report all 4 performance targets as 
fully met in September 2017, the full deployment date, when the 
equipment has been installed at all sites. 
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Navy—Consolidated Afloat 
Networks and Enterprise 
Services (CANES) 


Program description 

The CANES program is expected to consolidate and standardize the 
Navy’s existing network infrastructures and services by reducing and 
eliminating existing standalone afloat (i.e., surface ships and submarines) 
networks, providing a technology platform that can rapidly adjust to 
changing warfighting requirements, and reducing the hardware footprint 
on 259 afloat and maritime operations center platforms. 


Program essentials (as of October 2016) 

DOD component: Navy 
Program owner: C4I 

Prime contractors: Northrop Grumman, Serco, BAE Systems, DRS Laurel 
Technologies, General Dynamics, CGI Federal, Global Technical 
Systems 

Total number of contractors (as of May 2016): 82 


Table 6: Consolidated Afloat Networks and Enterprise Services Program costs 

(then-year dollars in millions) 


First acquisition 
program baseline 
January 2011 

Latest 
estimate 
July 2016 

Acquisition cost estimate 

Research, development, test, and evaluation 

$501.1 

$631.3 

Procurement 

3,476.1 

3,746.7 

Total acquisition cost 

$3,977.2 

$4,378.0 

Life-cycle cost estimate 

Operations and support 

8,763.7 

7,532.6 

Total life-cycle cost estimate 

$12,740.9 

$11,910.6 


Source: DOD data | GAO-17-322 
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Program status 

The CANES program had moved past the full deployment milestone. The 
program is expected to have all systems fully fielded by September 2024. 
According to program officials, risks remain that could result in delaying 
the fielding of CANES, thereby increasing risk to the program’s ability to 
achieve full deployment on schedule. 

Decreased planned cost estimate 

As of July 2016, the CANES program’s total life-cycle cost estimate was 
$11,910.6 million, a decrease of $830.3 million (-7 percent) from the 
original estimate of $12,740.9 million. In February 2016, in response to a 
full deployment decision, the life-cycle cost estimate was updated to 
reflect reductions in procurement costs, the program’s technology refresh 
schedule, and manpower costs. 

Exceeded planned schedule estimate 

The CANES program experienced a significant delay in reaching a full 
deployment decision. In June 2014, the program slipped the objective 
date for full deployment decision 2 years, from December 2013 to 
December 2015. A full deployment decision was achieved in October 
2015, and full deployment is estimated for September 2024. According to 
program officials, an unstable funding environment is the largest risk to 
achieving full deployment. 

Did not fully meet all of the system performance targets 

The CANES program had not met all 9 of the performance targets. 
Specifically, of the 9 performance targets, 8 have been met, while 1 
performance target was partially met—the net ready performance target. 
According to the program office, this performance target was only partially 
met because testing for afloat core services has been deferred. 
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Navy—Common Aviation 
Command and Control 
System Increment 1 
(CAC2S Inc 1) 


Program description 

The CAC2S Inc 1 program is expected to provide an integrated and 
coordinated modernization effort for the equipment of the Marine Air 
Command and Control System. The system is expected to provide 
enhanced capability for three air defense centers to support aviation 
employment in joint, combined, and coalition operations. It will provide 
tactical situational display, information management, sensor and data link 
interface, and operational facilities for planning and execution of Marine 
Aviation missions. 


Program essentials (as of October 2016) 

DOD component: Navy 

Program owner: USMC, Program Executive Officer, Land Systems 
Prime contractors: General Dynamics, C4 Systems, Inc. 

Total number of contractors (as of May 2016): 5 


Table 7: Common Aviation Command and Control System Increment 1 Program 
costs 

(then-year dollars in millions) 


First acquisition 
program baseline 
August 2000 

Latest 
estimate 
March 2016 

Acquisition cost estimate 

Research, development, test, and 
evaluation 


$559.3 

Procurement 


322.8 

Operations and maintenance 

Total acquisition cost 

$173.4 

$882.1 

Life-cycle cost estimate 

Operations and support 

173.6 

1,090.6 

Total life-cycle cost estimate 

$347.0 

$1,972.7 


Source: DOD data | GAO-17-322 
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Program status 

As of March 2016, Phase 1 of the CAC2S Inc 1 program was in the 
operations and sustainment acquisition phase and Phase 2 was in the 
production and deployment phase. The program has been rebaselined 
once following a critical change that occurred in 2009 due to the 
program’s inability to meet the initial operational capability date. 

Exceeded planned cost estimate 

As of March 2016, the CAC2S Inc 1 program’s total estimated life-cycle 
estimate was $1,972.7 million, an increase of $1,625.7 million (469 
percent) from the original life-cycle estimate of $347 million. Similarly, the 
program’s current total acquisition estimate is $882 million, an increase of 
$708.7 million (409 percent) from the original total acquisition estimate of 
$173.4 million. As we reported in March 2016, 2 factors that contributed to 
the cost increase were early challenges in estimating costs due to 
program scope growth and need for restructuring. Program officials noted 
funding instability as a potential area of risk in meeting cost estimates. 

Exceeded planned schedule estimate 

As of October 2015, the CAC2S Inc 1 program had a 13-year, 9-month 
slippage in their full deployment date due to a major change in project 
scope and restructuring of the project. 

Met all of the system performance targets 

CAC2S Inc. 1 had met both of the technical performance targets which 
were (1) net ready, and (2) data fusion. 


2 GAO-16-336 
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Defense Logistics 
Agency—Defense 
Agencies Initiative 
Increment 2 (DAI Inc 2) 


Program description 

The DAI Inc 2 program’s mission is expected to deliver auditable, 
accurate, timely, authoritative financial data to support DOD’s goal of 
standardizing financial management practices, improving financial 
decision support, and supporting audit readiness. Defense agencies use 
more than ten different non-compliant financial management systems 
supporting diverse operational functions and the warfighter in decision 
making and financial reporting. 

Program essentials (as of October 2016) 

DOD component: Defense Logistics Agency 
Program owner: OSD (C) 

Prime contractor: Credence Management Solutions, LLC; TeraThink 
Corp; IBM; CACI, Inc; Northrop Grumman Systems Corp; CACI ISS, Inc; 
Amyx, Inc 

Total number of contractors (as of May 2016): 25 


Table 8: Defense Logistics Agency- 
Program costs 

(then-year dollars in millions) 

-Defense Agencies Initiative Increment 2 


First acquisition 
program baseline 

July 2014 

Latest 
estimate 
March 2016 

Acquisition cost estimate 

Research, development, test, and 
evaluation 

$155.3 

$157.9 

Operation and maintenance 

27.0 

27.0 

Total acquisition cost 

$182.3 

$184.9 

Life-cycle cost estimate 

Operations and support 

683.9 

683.9 

Total life-cycle cost estimate 

$866.2 

$868.8 


Source: DOD data | GAO-17-322 
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Program status 

As of March 2016, the DAI Inc 2 program was in the limited deployment 
phase of the business capability life cycle. According to a 2016 MAIS 
annual report, the program was on track to remain within the schedule, 
cost, and performance thresholds identified in the original estimate. 
According to program officials, the program was rebaselined to adjust 
release 2 objective and threshold dates for limited fielding to 
accommodate the time frame for resolving a bid protest. Notwithstanding 
this additional time for the bid protest, the program achieved the original 
acquisition program baseline objective date for this event. 

Exceeded planned cost estimate 

As of March 2016, the DAI Inc 2 program’s total life-cycle estimate was 
$868.8 million, an increase of $2.6 million (<1 percent) from the original 
life-cycle cost estimate of $866.2 million. Similarly, the program’s current 
total acquisition estimate was $184.9 million, an increase of $2.6 million 
(1 percent) from the original acquisition estimate of $182.3 million. 
However, these cost increases were within the planned cost objective 
estimates. Officials cited funding instability as a potential risk to cost as 
the program moves forward. 

Met planned schedule estimate 

The DAI Inc 2 program had met the first acquisition program baseline 
objective schedule estimate for limited fielding in September 2015. 
Officials noted they were on schedule to stay within the threshold date of 
April 2017 for a full deployment decision. 

Did not fully meet all of the system performance targets 

According to the 2016 MAIS annual report, the DAI Inc 2 program had not 
met any of the five of the technical performance targets which are (1) 
financial system performance, (2) business enterprise architecture 
compliance, (3) support net centric DOD military operations, (4) managed 
in network, and (5) effectively exchange information. 
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Department of Defense— 
Healthcare Management 
System Modernization 
(DHMSM) 


Program description 

The DHMSM Program is expected to provide a configurable and scalable 
modernized electronic Health Record System. It will replace DOD’s 
legacy healthcare systems including the Armed Force’s Health 
Longitudinal Technology Application, Composite Health Care System 
(inpatient), and most components of the Theater Medical Information 
Program-Joint program, with a commercial Off-The-Shelf electronic health 
record system. It is expected to address the current state of the Military 
Health System, where multiple legacy systems and data stores, 
developed over decades, are in need of modernization. 


Program essentials (as of October 2016) 

DOD component: Defense Health Agency 

Program owner: PEO Defense Healthcare Management Systems 

Prime contractor: Leidos 

Total number of contractors (as of May 2016): 35 


Table 9: Department of Defense Healthcare Management System Modernization 
Program costs 

(then-year dollars in millions) 


First acquisition 
program baseline 

May 2016 

Latest 

estimate 

N/A 

Acquisition cost estimate 

Research, development, test, and 
evaluation 

$881.9 

N/A 

Procurement 

2372.7 

N/A 

Operations and maintenance 

480.5 

N/A 

Total acquisition cost 

$3,735.1 

N/A 

Life-cycle cost estimate 

Operations and support 

4,929.5 

N/A 

Total life-cycle cost estimate 

$8,664.6 

N/A 


Source: DOD data | GAO-17-322 


Cost, schedule, and performance summary 

Not applicable: The program obtained its first baseline on May 3, 2016. 
Due to the recent establishment of the program’s first baseline, a history 
of cost, schedule, and technical performance has not been established. 
Therefore, the cost, schedule, and technical performance is not 
discussed. 
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Air Force—Defense 
Enterprise Accounting 
and Management 
System-Increment 1 
(DEAMS Inc 1) 


Program description 

The DEAMS Inc 1 program is expected to provide the Air Force with an 
entire spectrum of financial management capabilities, including 
collections; commitments and obligations; cost accounting; general 
ledger; funds control; receipts and acceptance; accounts payable and 
disbursement; billing; and financial reporting. The DEAMS program is 
also intended to be a key component of DOD’s solution for achieving 
fully-auditable financial statements by September 30, 2017, as required 
by the National Defense Authorization Act for Fiscal Year 2010. 

Program essentials (as of October 2016) 

DOD component: Air Force 
Program owner: Air Force Financial Management 
Prime contractors: Accenture and Kearney 
Total number of contractors (as of May 2016): 29 


Table 10: Defense Enterprise Accounting and Management System-Increment 1 
Program costs 

(then-year dollars in millions) 


First acquisition 
program baseline 
February 2012 

Latest 
estimate 
July 2016 

Acquisition cost estimate 

Research, development, test, and 
evaluation 

$591.1 

$560.7 

Procurement 

50.0 

31.4 

Total acquisition cost estimate 

$641.1 

$592.1 

Life-cycle cost estimate 

Operations and support 

$640.6 

$813.7 

Other 

152.6 

157.9 

Total life-cycle cost estimate 

$1,434.3 

$1,563.7 


Source: DOD data | GAO-17-322 
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Program status 

As of June 2016, the DEAMS Inc 1 program was in the limited 
deployment phase. In March 2016, the program was approved for limited 
deployments at smaller major commands. Since then, the program has 
undergone a Critical Change due to schedule slippage of the threshold 
full deployment date of more than a year. However, the Critical Change 
report was not completed at the time of our review. Specifically, the 
DEAMS Inc 1 program was not demonstrating enough progress towards 
operational effectiveness, suitability, and survivability to reach the full 
deployment date within one year of the milestone objective. In addition, 
the program office identified requirements management and technical 
risks as areas that pose risks to successfully meeting schedule 
milestones. 

Exceeded planned cost estimate 

As of July 2016, the DEAMS Inc 1 program’s total life-cycle estimate was 
$1,563.7, an increase of $124.9 million (9 percent) from the original life- 
cycle estimate of $1,434.3 million. In addition, the program’s cost 
estimate decreased by $49.0 million (-8 percent) when compared to the 
original total acquisition estimate of $641.1. 

Exceeded planned schedule estimate 

The DEAMS Inc 1 program was undergoing a critical change due to 
schedule slippages. The full deployment date has been exceeded by 
more than a year. However, a Critical Change report had not yet been 
issued at the time of our review. 

Did not fully meet all of the system performance targets 

The DEAMS Inc 1 program had partially met the system performance 
targets. Specifically, 6 of the 9 performance targets have been met, while 
3 have not. 
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Global Combat Support 
System—Army Increment 
1 (GCSS-A Inc 1) 


Program description 

The GCSS-A Inc 1 program is a commercial off-the-shelf enterprise 
resources planning system that is expected to provide users with supply, 
maintenance, property, integrated materiel management, and 
management functionality and support. It uses non-developmental items 
made up of software and server hardware and has two components: 
GCSS-Army, which provides logistics and financial operations, and the 
Army Enterprise System Integration Program, which is expected to 
provide enterprise hub services, master data management, and business 
intelligence. 


Program essentials (as of October 2016) 

DOD component: Army 

Program owner: Assistant Secretary of the Army, Acquisition, Logistics 
and Technology 

Prime contractor: Northrop Grumman 

Total number of contractors (as of May 2016): Northrop Grumman 


Table 11: Global Combat Support System- 

(then-year dollars in millions) 

—Army Increment 1 Program costs 


First acquisition 
program baseline 
April 2013 

Latest 
estimate 
May 2016 

Acquisition cost estimate 

Research, development, test, and 
evaluation 

$997.4 

$1,059.7 

Procurement 

834.5 

873.0 

Total acquisition cost 

$1,831.9 

$1,932.7 

Life-cycle cost estimate 

Operations and support 

2,137.0 

1,977.7 

Total life-cycle cost estimate 

$3,968.9 

$3,910.4 


Source: DOD data | GAO-17-322 
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Program status 

As of March 2016, the GCSS-A Inc 1 program was in the production and 
deployment phase of the Defense Acquisition Management System. 
According to the 2016 Annual Report, the program was on track to remain 
within schedule, cost, and performance thresholds. However, the GCSS- 
A Inc 1 program realized a decrease of $16 million in funding that could 
potentially cause, in part, a six-month slip in fielding with a completion 
date of March 2018. 

Decreased planned cost estimate 

As of May 2016, the GCSS-A Inc 1 program’s total life-cycle estimate was 
$3,910.4 million, a decrease of $58.5 million (-1.5 percent) from the 
original life-cycle estimate of $3,968.9 million. The current total acquisition 
estimate was $1,932.7 million, an increase of $100.8 million (5.5 percent) 
from the original total acquisition estimate of $1,831.9 million. 

Exceeded planned schedule estimate 

The GCSS-A Inc 1 program reached the full deployment decision in 
December 2012, 10 months after the objective date of February 2012, 
and 4 months after the threshold date of August 2012. This represents a 
significant change from the original estimate of February 2012. The 
program office cited the potential for a 6-month slip in schedule to meet 
the full deployment date of September 2017. According to a MAIS 
summary report, the program conducted a trade-off analysis to minimize 
a 6-month slip in fielding, as a result of a $16 million decrement in 2016 
funding. Other challenges in meeting schedule include requirements 
management, insufficient human resources, and the possibility for Army 
schedule changes. However, the program reports they are on schedule 
for full deployment prior to the threshold schedule date of March 2018. 

Met all of the system performance targets 

The GCSS-A Inc 1 program had met both of the systems performance 
targets which were (1) net ready, and (2) mission critical functions. 
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Marine Corps—Global 
Combat Support System- 
Marine Corps Logistics 
Chain Management 
Increment 1 (GCSS-MC 
LCM Inc 1) 


Program description 

The GCSS-MC LCM Inc 1 program is a portfolio of systems that is 
expected to support logistics elements of command and control, joint 
logistics interoperability, and secure access to and visibility of logistics 
data. As a program within GCSS-MC, Increment 1 is based on the 
implementation of Oracle e-Business Suite 11 i as the core software 
package. It is expected to provide the foundation for all future Marine 
Corps logistics systems modernization. 


Program essentials (as of October 2016) 

DOD components: Department of the Navy, United States Marine Corps 
Program owner: Deputy Commandant Installations and Logistics 
Headquarters Marine Corp 

Prime contractor: N/A, managed by the federal government 
Total number of contractors (as of May 2016): Not applicable 


Table 12: Global Combat Support System-Marine Corps Logistics Chain 

Management Increment 1 Program costs 

(then-year dollars in millions) 


First acquisition 
program baseline 

June 2007 

Latest 
estimate 
March 2016 

Acquisition cost estimate 

Research, development, test, and 
evaluation 

$102.3 

$307.9 

Procurement 

52.5 

88.3 

Operations and maintenance 

39.6 

160.5 

Total acquisition cost 

$194.4 

$556.7 

Life-cycle cost estimate 

Operations and support 

306.6 

1,265.0 

Total life-cycle cost estimate 

$461.4 

$1,821.7 


Source: DOD data | GAO-17-322 
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Program status 

As of March 2016, the GCSS-MC LCM Inc 1 program had been fully 
implemented and reported as a closed acquisition program. 

Exceeded planned cost estimate 

As of March 2016, the GCSS-MC LCM Inc 1 program’s total life cycle 
cost estimate was $1,821.70 million, an increase of $1,360.3 million (295 
percent) from the original total life cycle cost estimate of $461.4 million. 
The GCSS-MC LCM Inc 1 program’s current total acquisition cost 
estimate was $556.7 million, an increase of $362.3 million (186 percent) 
from the original estimate of $194.4 million. 

Exceeded planned schedule estimate 

The GCSS-MC LCM Inc 1 program had a significant change in reaching 
full deployment. The program reached the full deployment date in 
December 2015. When compared to the original date of May 2010 for full 
operational capability (equivalent is full deployment), this resulted in a 5 
year and 7 month delay. The program is fully implemented. 

Met all of the system performance targets 

As of March 2016, the GCSS-MC Inc 1 program had met both of the 
performance targets which were (1) Net ready, and (2) Visibility 
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Defense Health Agency— 
Integrated Electronic 
Health Record Increment 
1 (iEHR Inc 1) 


Program description 

The iEHR Inc 1 program is expected to deliver two user capabilities—a 
medical single sign on that streamlines the login process to allow users to 
sign in once and leverage securely stored credentials to automatically 
access the other available applications, and a context management 
capability that will automatically present a patient’s data within all 
applications. Other deliverables include enhanced infrastructure services, 
a development test center/environment configuration, and clinical data 
repository upgrades. 


Program essentials (as of October 2016) 

DOD component: Defense Health Agency 
Program owner: PEO DHMS 

Prime contractor: General Dynamics Information Technology 
Total number of contractors (as of May 2016): Not applicable 


Table 13: Integrated Electronic Health Record Increment 1 Program costs 

(then-year dollars in millions) 


First acquisition 
program baseline 

Feb 2013 

Latest 
estimate 
March 2015 

Acquisition cost estimate 

Research, development, test, and 
evaluation 

$69.8 

$46.7 

Procurement 

122.8 

71.2 

Acquisition operations and maintenance 

173.7 

131.4 

Total acquisition cost 

$366.3 

$249.3 

Life-cycle cost estimate 

Operations and support 

659.6 

58.3 

Total life-cycle cost estimate 

$1,025.9 

$307.6 


Source: DOD data | GAO-17-322 
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Program status 

As of March 2016, the iEHR Inc 1 program was in the sustainment phase 
and, according to a 2016 MAIS Annual Report, the program is closed out 
as an acquisition program. 

Decreased planned cost estimate 

The iEHR Inc 1 program’s March 2015 estimate for total life-cycle cost 
was $307.6 million, a decrease of $718.3 million (-70 percent) from the 
original total life-cycle estimate of $1,025.9 million. Likewise, the 
program’s total estimated acquisition cost was $249.3 million, a decrease 
of $117 million (-32 percent) from the original total acquisition cost 
estimate of $366.3 million. The reduction in estimated cost was primarily 
because of reductions in the program’s scope. 

Exceeded planned schedule estimate 

The iEHR Inc 1 program’s original objective schedule estimate for full 
deployment decision was December 2013. When compared to the actual 
full deployment decision date of November 2014 with the original date, 
the program’s schedule slipped 11 months. The program achieved full 
deployment in April 2015, 1 month ahead of schedule. 

Met all of the system performance targets 

The iEHR Inc 1 program had met all three of the performance targets 
which were (1) essential business function 1-context management, (2) 
single sign on, and (3) roaming 


Page 52 


GAO-17-322 DOD Major Automated Information Systems 




Appendix II: Profiles of Selected 
MAIS Programs 


Army—Integrated 
Personnel and Pay 
System—Army Increment 
1 (IPPS-AInc 1) 


Program description 

The IPPS-A Inc 1 program is expected to provide a 24-hour, web-based, 
integrated human resources system to soldiers, human resource 
professionals, combatant commanders, personnel and pay managers, 
and other authorized users. Specifically, the system is expected to 
provide a single, multi-component, trusted database with a single record 
for all Army soldiers, and serve as a trusted data source for personnel 
and human resources data for the entire Army. Soldiers will have web- 
based access to their personnel data, enabling them to better manage 
their careers and ensure accuracy of information. The system will allow 
for interface communications and new multi-component reports that 
include a Soldier Record Brief for all Army components (Active, National 
Guard and Reserve). 


Program essentials (as of October 2016) 

DOD component: Department of the Army 

Program owner: Department of the Army, Deputy Chief of Staff, G-1 
Prime contractors: EDC Consulting, LLC - System Integration, Booz Allen 
Hamilton-Support Services 

Total number of contractors (as of May 2016): N/A: Program is in 
sustainment 


Table 14: Integrated Personnel and Pay System—Army Increment 1 Program costs 

(then-year dollars in millions) 


First acquisition 
program baseline 
March 2012 

Latest 
estimate 
January 2015 

Acquisition cost estimate 

Research, development, test, and 
evaluation 

$154.8 

$190.9 

Procurement 

2.2 

.5 

Total acquisition cost 

$157.0 

$191.4 

Life-cycle cost estimate 

Operations and support 

201.4 

177.5 

Total life-cycle cost estimate 

$358.4 

$368.9 


Source: DOD data | GAO-17-322 
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Program status 

As of March 2016, the IPPS-A Inc 1 program had achieved full 
deployment and was in the sustainment phase. The program was re¬ 
baselined after implementing strategic recommendations by an 
independent review team established in June 2013 that determined the 
system design was sufficient to meet Increment I requirements. The re¬ 
baselined acquisition program baseline was signed by the Milestone 
Decision Authority in February 2015. 

Exceeded planned cost estimate 

As of January 2015, the IPPS-A Inc 1 program’s total estimated life-cycle 
cost was $368.9 million, an increase of $10.5 million (3 percent) from the 
original total estimated life-cycle cost of $358.4 million. The total 
estimated acquisition cost was $191.4 million (as of February 2015), an 
increase of $34.4 million (22 percent) from the original total acquisition 
cost of $157 million. According to program officials, the increase in cost 
was a direct result of deferring continuity of operations requirements and 
capturing the testing environment requirement under Increment 2. 

Exceeded planned schedule estimate 

The IPPS-A Inc 1 program achieved Milestone C and Full Deployment 
Decision about 6 months after the original threshold dates in August 2013 
and October 2013, respectively. During the acquisition, the delay 
associated with the slip for a Milestone C decision triggered a significant 
change to the program’s schedule. The program has achieved full 
deployment and is now in the sustainment phase. 

Met all of the system performance targets 

The IPPS-A Inc 1 program had met all three of the performance targets 
which were (1) net ready, (2) interoperability, and (3) availability 
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Army—Integrated 
Personnel and Pay 
System—Army Increment 
2 (IPPS-AInc 2) 


Program description 

The IPPS-A Inc 2 program is expected to provide a 24-hour, web-based, 
integrated Human Resources system for Army personnel. It is to deliver 
an integrated personnel and pay services for all Army components 
(Active, National Guard, and Reserve), building on a trusted database 
delivered by the IPPS-A inc 1 program. The IPPS-A Inc 2 program is 
planned to improve the Army’s financial management processes in 
support of military personnel and payroll by linking the personnel and 
payroll functions for all Army personnel, eliminating duplicate data entry, 
reducing complex system maintenance, and minimizing pay 
discrepancies. The IPPS-A Inc 2 program is expected to account for 
status changes between Active, Reserve, and National Guard 
components to ensure accurate service time, minimize impact on 
individual pay, credit for service, and other benefits. 


Program essentials (as of October 2016) 

DOD component: Department of the Army 

Program owner: Department of the Army, Deputy Chief of Staff, G-1 
Prime contractor: CACI-System Integration, Booz Allen Hamilton-Support 
Services 

Total number of contractors (as of May 2016): 38 


Table 15: Integrated Personnel and Pay System—Army Increment 2 Program costs 

(then-year dollars in millions) 


First acquisition 
program baseline 
March 2015 

Latest 
estimate 
March, 2016 

Acquisition cost estimate 

Research, development, test, and 
evaluation 

$644.3 

$709.7 

Procurement 

81.5 

110.5 

Total acquisition cost 

$725.8 

$820.2 

Life-cycle cost estimate 

Operations and support 

1,025.8 

1,261.7 

Total life-cycle cost estimate 

$1,751.0 

$2,081.9 


Source: DOD data | GAO-17-322 
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Program status 

As of March 2016, the IPPS-A Inc 2 program is in the development and 
deployment phase of the defense acquisition life cycle. A Milestone B 
decision was achieved in December 2014. In fiscal year 2016, the 
program completed several reviews—system requirements, system 
functional, and integrated baseline. The program indicated requirements 
growth as an area of potential risk. 

Exceeded planned cost estimate 

As of March 2016, the IPPS-A Inc 2 program’s total life-cycle cost 
estimate was $2,081.9 million, an increase of $330.9 million (19 percent) 
from the original estimated life-cycle cost of $1,751.0 million. The current 
total estimated acquisition cost estimate was $820.2 million, an increase 
of $94.3 million (13 percent) from the original estimated acquisition cost of 
$725.8 million. 

Met planned schedule estimate 

The IPPS-A Inc 2 program achieved Milestone B on 19 December 2014, 
within the planned threshold schedule. 

Not yet applicable 

The IPPS-A Inc 2 program was still early in development and system 
performance data was not available. 
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Air Force—Integrated 
Strategic Planning and 
Analysis Network 
Increment 4 (ISPAN Inc 4) 


Program description 

The ISPAN Inc 4 program is expected to be a system-of-systems that 
spans multiple security enclaves for strategic and operational level 
planning and leadership decision making. The system is composed of two 
primary components: (1) a global adaptive planning collaborative 
information environment that manages strategy-to-execution planning 
across all United States Strategic Command (USSTRATCOM) mission 
areas and (2) a mission planning and analysis system that supports 
developing nuclear and conventional force application plans. Both 
components establish a framework to support the USSTRATCOM’s 
effects-based planning and analysis activities. 


Program essentials (as of October 2016) 

DOD component: Air Force 
Program owner: USSTRATCOM 

Prime contractors: BAE, Leidos, Lockheed Martin, and Northrop 
Grumman 

Total number of contractors (as of May 2016): 19 


Table 16: Integrated Strategic Planning and Analysis Network Increment 4 Program 
costs 

(then-year dollars in millions) 


First acquisition 
program baseline 
August 2014 

Latest 
estimate 
June 2016 

Acquisition cost estimate 

Research, development, test, and 
evaluation 

$149.3 

$149.3 

Acquisition O&M 

$13.2 

$13.2 

Total Acquisition Cost 

$162.5 

$162.5 

Life cycle cost estimate 

Operations and support 

$24.7 

$24.7 

Total life cycle cost estimate 

$187.2 

$187.2 


Source: DOD data | GAO-17-322 
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Program status 

As of June 2016, the ISPAN Inc 4 program was in the engineering and 
manufacturing development phase of the Defense Acquisition 
Management System. According to officials, the program was on track to 
remain within the schedule, cost, and performance thresholds identified in 
the original estimate, with no major risks. The program has not been 
rebaselined. 

Met planned cost estimate 

The ISPAN Inc 4 program’s current cost estimates have not changed 
since their first estimate. 

Met planned schedule estimate 

The ISPAN Inc 4 program’s current schedule estimates have not changed 
since their first estimate 

Not yet applicable 

The ISPAN Inc 4 program was still early in development and system 
performance data was not yet available. 
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Air Force—Joint Space 
Operations Center, 

Mission System Increment 
2 (JMS Inc 2) 


Program description 

The JMS Inc 2 Program is expected to replace the legacy Space Defense 
Operations Center hardware and services with extensible and sustainable 
infrastructure. The effort is planned to integrate components of the Space 
Situational Awareness mission applications and Command and Control 
capabilities into the Joint Space Operations Center to create timely, 
actionable knowledge necessary for maintaining space superiority and 
exercising command and control capabilities of space forces. 


Program essentials (as of October 2016) 

DOD component: Air Force 

Program owner: Air Force Space Command 

Prime contractor: N/A, managed by federal government 

Total number of contractors (as of May 2016): 7 


Table 17: Joint Space Operations Center, Mission System Increment 2 Program 
costs 

(then-year dollars in millions) 


First acquisition 
program baseline 

June 2013 

Latest 
estimate 
March 2016 

Acquisition cost estimate 

Research, development, test, and 
evaluation 

$312.7 

$462.8 

Total acquisition cost 

$312.7 

$462.8 

Life-cycle cost estimate 

Operations and sustainment 

787.8 

503.1 

Total life-cycle cost estimate 

$1,100.5 

$965.9 


Source: DOD data | GAO-17-322 
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Program status 

As of March 2016, the JMS Inc 2 program was in the engineering and 
manufacturing development acquisition phase. Due to issues associated 
with schedule delays of more than 12 months and acquisition 
development costs increasing more than 25 percent, the program 
underwent a critical change. A critical change report was delivered to 
Congress in September 2016 due to the delayed completion of the 
program’s Milestone C and full deployment decision by more than 12 
months. According to the report, three factors that caused a critical 
change were (1) the program’s original schedule was aggressive and 
contained a significant level of concurrency, (2) funding and manpower 
issues, and (3) contracting issues. 

Decreased planned cost estimate 

As of March 2016, the JMS Inc 2 program’s total life-cycle cost estimate 
was $965.9 million, a decrease of $134.6 million (-12 percent) from the 
original total estimate of $1,100.5 million. The latest estimate for total 
estimated acquisition cost was $462.8 million, an increase of $150.1 
million (48 percent) from the original acquisition cost estimate of $312.7 
million. The decrease for the total life-cycle cost estimate was because 
the original estimate for operations and sustainment costs was 20 years 
after fielding while the new estimate is for 10 years after fielding. 

Exceeded planned schedule estimate 

Due to significant delays in the JMS Inc 2 program’s aggressive schedule, 
the program underwent a rebaseline effort and issued a critical change 
report in September 2016. The new estimate for a full deployment 
decision is May 2019. With an original schedule date of July 2016, this 
represents a 2-year, 10-month slippage. 

Not yet applicable 

The JMS Inc 2 program was still early in development and system 
performance data was not yet available. 
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Army—Logistics 
Modernization Program 
Increment 2 (LMP Inc 2) 


Program description 

The LMP Inc 2 program is expected to deliver an enterprise solution that 
builds, sustains, and generates warfighting capabilities using a fully- 
integrated supply chain and maintenance, repair, and overhaul system. 
The LMP Inc 2 program’s support is critical to the Army’s ability to 
achieve an integrated system that enables materiel readiness and asset 
management and accountability, architecture, and acquisition compliancy. 
When deployed, it is expected to deliver capabilities that address critical 
Army Materiel Command requirements for automation of the industrial 
base shop floor operations, standardization of ammunition automatic 
identification technology, non-Army managed items management, Army 
prepositioned stock planning, national maintenance program, and the 
expansion and refinement of the integration between Army enterprise 
resource planning systems. 


Program essentials (as of October 2016) 

DOD component: Army 

Program owner: PEO EIS 

Prime contractor: CSRA, LLC 

Total number of contractors (as of May 2016): 188 


Table 18: Logistics Modernization Program Increment 2 Program costs 

(then-year dollars in millions) 


First acquisition 
program baseline 

Aug 2013 

Latest 
estimate 
July 2016 

Acquisition cost estimate 

Army working capital funds-capital 
investment program 

$344.7 

$370.8 

Army working capital funds-operations 

60.4 

48.4 

Total acquisition cost 

$405.1 

$419.2 

Life-cycle cost estimate 

Operations and support 

$324.8 

$317.6 

Total life-cycle cost estimate 

$729.9 

$736.8 


Source: DOD data | GAO-17-322 
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Program status 

As of June 2016, the LMP Inc 2 program was in the operation and 
support phase of the acquisition life cycle. The program went to full 
deployment in September 2016. The program will continue to provide 
national level logistics and financial operations to Army arsenals and 
depots. 

Exceeded planned cost estimate 

As of July 2016, the LMP Inc 2 program’s total estimated life-cycle cost 
estimate was $736 million, an increase of $6.9 million (1 percent) from the 
original total life-cycle cost estimate of $729.9 million. Likewise, the total 
current estimated acquisition cost estimate was $419.2 million, an 
increase of $14.1 million (3 percent) from the original estimate of $405.1 
million. However, these increases were within the threshold estimate 
established for the program. 

Exceeded planned schedule estimate 

The LMP Inc 2 program had a schedule slip of 5 months when comparing 
the original objective full deployment decision date of October 2015 to the 
current estimate of July 2016. However, the schedule slip was within the 
established threshold date of April 2016. 

Met all of the system performance targets 

The LMP Inc 2 program had met all seven of the system performance 
targets which were (1) support net-centric military operations, (2) enter 
and be managed in the network, (3) net readiness exchange information, 
(4) survivability system information assurance, (5) survivability information 
assurance system disaster recovery, (6) sustainment operational 
availability, and (7) training proficiency level. 
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Defense Information 
Systems Agency— 
Teleport Generation 3 
(Teleport Gen 3) 


Program description 

The Teleport Gen 3 program is expected to be a satellite communications 
gateway that will enable the transportation of voice, video, and data 
information to deployed forces, and links the deployed warfighter to the 
sustaining base. The system is expected to provide expanded system 
capacity, throughput, and functional capabilities to greatly enhance 
support to tactical and deployed warfighters worldwide. 

Program essentials (as of October 2016) 

DOD component: Defense Information Systems Agency 
Program owner: DISA Component Acquisition Executive 
Prime contractors: Booz Allen Hamilton, Femme Comp Inc., and Itility 
Total number of contractors (as of May 2016): 5 


Table 19: Teleport Generation 3 Program costs 

(then-year dollars in millions) 


First acquisition 
program baseline 
September 2010 

Latest 
estimate 
July 2016 

Acquisition cost estimate 

Research, development, test, and 
evaluation 

$19.20 

$18.30 

Procurement 

243.30 

229.60 

Total acquisition cost 

$262.50 

$260.30 

Life-cycle cost estimate 

Operations and support 

301.20 

315.00 

Total life-cycle cost estimate 

$563.70 

$575.30 


Source: DOD data | GAO-17-322 
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Program status 

As of July 2016, the Teleport Gen 3 program was in the production and 
deployment acquisition phase. 

Exceeded planned cost estimate 

As of July 2016, the Teleport Gen 3 program’s current total life-cycle cost 
estimate was $575.3 million, an increase of $11.6 million (2 percent) from 
the original estimate of $563.7 million. 

Exceeded planned schedule estimate 

The Teleport Gen 3 program had a schedule slip of 3 years and 2 
months, from January 2013 to March 2016 when comparing the original 
baseline Milestone C (Gen 3, Phase 3) objective schedule estimate. 

Did not fully meet all of the system performance targets 

The Teleport Gen 3 program’s Phase 1 performance target had been 
met. However, performance targets for Phases 2 and 3 had not been met. 
According to the program office, all developmental and operational testing 
had been completed for Teleport Gen 3 Phases 1 and 2. Additional 
testing is planned for Phase 3. 
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Program description 

The TMC program is a suite of systems that is expected to provide 
mission command computing capabilities to Army commanders and their 
staffs, consisting of a user-customizable common operating picture 
enabled with real-time collaboration. These capabilities, together with 
voice-over-internet-protocol, will provide real-time situational awareness 
and decision support across the formation (battalion thru Army Service 
Component Command). A component of the mission command capability 
is the ability to layer information from multiple functions on the same map 
display and share among command posts, allowing all parties to have a 
common view. 3 

Program essentials (as of October 2016) 

DOD component: Army 
Program owner: Mission Command 

Prime contractor: General Dynamics (previous); currently a government 
led effort by Army Research and Development Engineering Center 
Total number of contractors (as of May 2016): N/A. Army is the lead 


Table 20: Tactical Mission Command Program costs 

(then-year dollars in millions) 


First acquisition 
program baseline 
July 2005 

Latest 
estimate 
March 2016 

Acquisition cost estimate 

Research, development, test, and 
evaluation 

$319.5 

$439.3 

Procurement 

500.9 

1,752.8 

Total acquisition cost 

$820.4 

$2,192.1 

Life-cycle cost estimate 

Operations and support 

73.4 

123.4 

Total life-cycle cost estimate 

$893.8 

$2,315.5 


Source: DOD data | GAO-17-322 


3 While these capabilities are currently provided by Command Post of the Future, the 
Tactical Mission Command system is transitioning to the web-based Command Post 
Computing Environment. 
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Program status 

As of March 2016, the TMC program is currently post Milestone C. The 
program has been rebaselined two times. A critical change report was 
issued in response to a cost breach caused by a growth in the program’s 
scope growth that was considered a “fact of life” change. As of March 
2016, program officials reported they are on track to remain within the 
schedule, cost, and performance thresholds identified in the original 
estimate. 

Exceeded planned cost estimate 

As of March 2016, the TMC program’s total life cycle cost estimate was 
$2,315.50 million, an increase of $1,421.7 million (159 percent) from the 
original total life cycle cost estimate of $893.80 million. The TMC 
program’s latest total acquisition cost estimate was $2,192.1 million, an 
increase of $1,371.7 million (167 percent) from the original acquisition 
estimate of $820.40 million in March 2016. 4 We reported in March 2016 
that the Army’s TMC program estimated program development costs had 
increased by 45 percent over the original estimate due to program scope 
changes derived from the realignment of Command Post of the Future as 
a foundation for Mission Command Collapse, the integration of 
Personalized Assistant that Learns, and the incorporation of future force 
requirements. According to a 2016 MAIS Annual Report, the program was 
on track to remain within the schedule, cost, and performance thresholds 
identified in the original estimate. 

Met planned schedule estimate 

As we reported in our March 2016 report, the TMC program experienced 
a slight slippage in schedule (3 months), however, it was within the 
established threshold date. Program officials report they are still on track 
to reach full deployment by December 2018. 

Met all of the system performance targets 

As of March 2016, the TMC program had met all three of the performance 
targets which were (1) net ready, (2) display the common operational 
picture, and (3) disseminate (create and exchange) orders. 


4 GAO-16-336. 
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DOD—Theater Medical 
Information Program— 
Joint Increment 2 


(TMIP-J Inc 2) 


Program description 

The TMIP-J Inc 2 program is expected to integrate components of the 
Military Health System sustaining base systems and the Services' 
medical information systems to ensure timely interoperable medical 
support for mobilization, deployment, and sustainment of all theater and 
deployed forces. The system is expected to enhance clinical care and 
information capture at all levels of care in theater, transmits critical 
information to the theater commander; the evacuation chain for combat 
and non-combat casualties; and forges the theater links of the health 
record to the sustaining base and the Department of Veterans Affairs. 
The TMIP-J program is the medical component of the Global Combat 
Support System. 

Program essentials (as of October 2016) 

DOD components: DOD, Joint Participants: U.S. Army; U.S. Navy and 
Marine Corps; U.S. Air Force 

Program owner: PEO Defense Healthcare Management Systems 
Prime contractor: KRATOS Inc. 

Total number of contractors (as of April 2016): 6 


Table 21: Theater Medical Information Program—Joint Increment 2 Program costs 

(then-year dollars in millions) 


First acquisition 
program baseline 
December 2007 

Latest 
estimate 
July 2016 

Acquisition cost estimate 

Research, development, test, and 
evaluation 

$124.2 

$323.7 

Procurement 

8.6 

16.7 

Operations and maintenance 

100.5 

267.8 

Total acquisition cost 

$233.3 

$608.2 

Life-cycle cost estimate 

Operations and support 

442.8 

448.2 

Total life-cycle cost estimate 

$676.1 

$1,056.4 


Source: DOD data | GAO-17-322 
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Program status 

As of July 2016, the TMIP-J Inc 2 program was reported as fully deployed 
and was in the sustainment phase. The system consists of two 
increments—Increment 1 was fielded in 2003, and development of 
Increment 2 was being done in three releases. Release 1 was fielded in 
2009 and Release 2 was fielded in 2013. Release 3, is the last major 
release. In February 2016, the Under Secretary of Defense for 
Acquisition, Technology, and Logistics declared that the TMIP-J Inc 2 
program had met its full deployment decision action items. He delegated 
Milestone Decision Authority to the Program Executive Officer, Defense 
Healthcare Management Systems, and authorized the Services to declare 
full deployment in accordance with their Service acquisition strategies. 

Exceeded planned cost estimate 

As of July 2016, the TMIP-J Inc 2 program’s total life cycle cost estimate 
was $1,056.4 million, an increase of $380.3 million (56 percent) from the 
original estimate of $676.1 million. The total acquisition cost estimate was 
$608.2 million, an increase of $374.9 million (161 percent) from the 
original estimate of $233.3 million. 

Exceeded planned schedule estimate 

The TMIP-J Inc 2 program had a 5-year, 6-month change to the full 
deployment decision date when a breach occurred. Program officials 
stated the schedule change was due to a mandate to increase the scope 
of the program. In turn, a revised acquisition program baseline was 
approved in January 2012. The objective date for full deployment 
changed from June 2008 to December 2013, and a threshold date 
changed from December 2008 to June 2014. The TMIP-J Inc 2 program 
reached full deployment in May 2016 and has transitioned into the 
sustainment phase. 

Met all of the system performance targets 

The TMIP-J Inc 2 program had met all ten of the performance targets 
which were (1) net ready, (2) data transfer data availability, (3) data 
availability, currency, and responsiveness—medical logistics visibility, (4) 
data availability, currency and responsiveness—patient visibility, (5) data 
availability, currency and responsiveness—medical infrastructure 
readiness, (6) data availability, currency and responsiveness—individual 
medical readiness, (7) data availability, currency and responsiveness— 
illness and injury rates, (8) data availability, currency and 
responsiveness—reportable medical events, (9) standards compliance, 
and (10) system operational availability and responsiveness. 
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ACQUISITION 


OFFICE OF THE ASSISTANT SECRETARY OF DEFENSE 
3000 DEFENSE PENTAGON 
WASHINGTON. DC 20301-3000 


MAR 2 3 2017 


Ms. Carol C. Harris 

Director, Information Technology Acquisition Management Issues 
U.S. Government Accountability Office 
441 G Street, NW 
Washington DC 20548 

Dear Ms. Harris, 

This is the Department of Defense (DoD) response to the GAO Draft Report 
GAO-17-322, “DOD MAJOR AUTOMATED INFORMATION SYSTEMS: 


Improvements Can Be Made in Applying Leading Practices for Managing Risk and 
Testing," dated February 24, 2017 (GAO Code 100780). Responses to the report 


recommendations are attached. 



Performing the Duties of the 
Assistant Secretary of Defense 
for Acquisition 


Enclosure: 
As stated 
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GAO DRAFT REPORT DATED FEBRUARY 24, 2017 
GAO-17-322 (GAO CODE 100780) 

“DOD MAJOR AUTOMATED INFORMATION SYSTEMS: IMPROVEMENTS CAN 
BE MADE IN APPLYING LEADING PRACTICES FOR MANAGING RISK AND 

TESTING” 

DEPARTMENT OF DEFENSE COMMENTS 
TO THE GAO RECOMMENDATION 


RECOMMENDATION 1: To help improve the management of DOD's MAIS programs, GAO 
is recommending that the Secretary of Defense take the following action: direct the Secretary of 
the Army to direct the Product Manager for Global Combat Service Support - Army (GCSS- 
Army) Increment 1 to establish standard operating procedures for managing risks that include 
guidance for establishing thresholds and bounds for key risk areas. 

DoD RESPONSE: Concur. GCSS-Army Increment 1 is already using an established standard 
operating procedure for managing risk which includes the concepts of‘‘thresholds” and 
“bounds.” As a DoD organization, GCSS-Army adheres to the standards articulated in the DoD 
Risk, Issue, and Opportunity Management Guide (RIO). The leading practices identified on 
Table 3 of the report (attributed to the Software Engineering Institute’s Capability Maturity 
Model for Acquisition (CMMI-ACQ)) are parallel to the identified practices in the RIO. While 
there is minor variation in terminology—CMMI-ACQ is focused commercially and DoD RIO is 
focused on Defense acquisition—the intent to manage risk through a framework of thresholds 
and bounds is consistent across both models. GCSS-Army is successfully applying the leading 
practices as part of its documented risk management process. 

RECOMMENDATION 2: To help improve the management of DOD's MAIS programs, GAO 
is recommending that the Secretary of Defense take the following action: direct the Secretary of 
the Air Force to direct the program manager for AOC-WS Inc 10.2 to develop an overall risk 
mitigation plan to guide the implementation of individual risk mitigation and contingency plan 
activities. 

DoD RESPONSE: Concur. While the AOC WS System Program Office (SPO) has a draft Risk 
and Opportunities Management (ROM) Plan it has neither been published nor rigorously 
adhered to. One of the SPO’s lessons learned and process improvement initiatives coming out of 
the AOC 10.2 Critical Change was to revamp ROM practices in the SPO. The Program Manager 
has taken steps to address this need including re-initiating weekly ROM meetings, both internally 
and jointly, with the prime contractor and reviewing program risks bi-weekly with the Program 
Executive Officer. The ROM Plan will be published by 3 Apr 17. 


1 


Page 70 


GAO-17-322 DOD Major Automated Information Systems 





Appendix III: Comments from the Department 
of Defense 


RECOMMENDATION 3: To help improve the management of DOD's MAIS programs, GAO 
is recommending that the Secretary of Defense take the following action: direct the Secretary of 
the Air Force to direct the program manager for JMS to appoint a chief developmental tester to 
oversee systems testing and integration activities. 

DoD RESPONSE: Concur. The JMS program has identified and selected a qualified individual 
to fill the Chief Developmental Test position. The new Developmental Test Chief will be in 
place in July 2017. 
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